How to fork a go library (my way)

  • Git fork
  • Go mod init if not yet a module
  • Update path to new module github.com/someone/library -> github.com/you/library. This is because the module will continue to look for dependencies in the old path
  • Bump version with a tag (v1.0.0 -> v1.0.1)
  • Push the tag to Git
  • Go mod tidy in any project using the go library

Prune big files from Git with BFG

Step 2: Clone your repo as a mirror

Run this to clone your repo as a mirror.

git clone --mirror git://example.com/your-repo.git

The —-mirror flag means that it doesn’t actually download any of the files from your codebase. It downloads the .git directory contents into the top level, so it looks a little funky at first.

Step 3: Back up your repo

Just in case, now is a good idea to back up your repo to a separate location so that you can restore from it if needed.

cp -r your-repo.git your-repo-backup.git

Step 4: Run BFG to remove large blobs

In BFG, you have 3 main options for clearing out large files from your Git history.

Note: “blobs” are the objects that store file contents in Git.

Option 1: Strip blobs bigger than a specified size

In this case, you’re telling BFG to remove any blobs from history that are larger than a specified size. For example, to strip out any blobs bigger than 10MB, you run:

bfg --strip-blobs-bigger-than 10M your-repo.git

Option 2: Strip the biggest blobs, limited to a specified number

If you’d rather just strip the biggest blobs regardless of how large they are, BFG makes that easy:

bfg --strip-biggest-blobs 100 your-repo.git

This command will strip the 100 largest blobs from your repo history.

Option 3: Strip specific blobs, specified by IDs

If you know of specific files you want to get rid of, you can take this approach. In this case, you’ll pass in a text file that contains the list of blob IDs you want to strip.

bfg --strip-blobs-with-ids blobs.txt your-repo.git

You can use a one-liner like this like this to find the IDs of the largest blobs.

Protecting specific branches

BFG is very good at making sure that it doesn’t touch any of the files on your HEAD. It only adjusts your history, not the files in your latest commit.

But if you have other branches you want to make sure you protect, you can do that with the --protect-blobs-from flag:

bfg --protect-blobs-from master,dev,stage --strip-biggest-blobs 100 your-repo.git

Step 5: Expire and prune your repo

You’re removing things with BFG. Now it’s time to expire and prune your git history to reflect your changes. Here’s how you’d do that:

cd your-repo.git git reflog expire --expire=now --all && git gc --prune=now --aggressive

Note that this command can take a long time depending on your repo size. So it’s best to make sure you’re done running BFG first. You only want to have to run this command one time.

Step 6: Check your repo size

Verification time! Did the repo shrink? If not, then you probably missed a step or didn’t remove the correct files.

Step 7: Push your changes

Now you’re ready to push all your updates so that everyone can enjoy the new, slimmer repo you have fashioned.

git push

And that’s that! Your repo should be smaller, without any noticeable negative effects. Thanks BFG!

source: https://www.phase2technology.com/blog/removing-large-files-git-bfg

Gumana Series – Understanding Fabric’s Ordering

Introduction to Gumana

Gumana is a portmanteau of 금요일 (Friday) 만나다 (Meeting), a series of technical talk every Friday at Medium

About this talk

A visual explanation of Kafka’s Ordering Design, why does it need a queue, kafka, synchronization between Orderer Nodes

Presentation slide

Presentation video

Medium interview questions

Based on Korean Version 1.9.6 (https://hamait.tistory.com/1054), last updated October 2019.

You are not expected to be 100% knowledgeable about those, but instead show your depth in understanding what you have experience with or is interested in. Focus on what you know well.

Part 1: Blockchain

What are Double spending, Replay attack, Eclipse attack

Part 2: Bitcoin

  • How can we ensure the integrity of Bitcoin transactions? How do you trust the previous output in the input of the next transaction?
  • What’s bloom filter SPV in Bitcoin?

Part 3: Ethereum

  • What’s the difference between Transaction and Raw Trasaction?
  • What’s nonce in an Ethereum transaction? Why is there no nonce in Bitcoin?

Part 4: Hyperledger fabric

  • Explain the transaction flow of Hyperledger fabric
  • What is MVCC Collision and Optimistic Lock on Hyperledger Fabric?
  • What is MSP in Hyperledger Fabric
  • What are channel MSPs and network MSPs in a Hyperledger fabric?
  • What’s nonce in Hyperledger fabric. What is the difference with Ethereum’s?
  • How events are created in Hyperledger fabric, how can the client know about an event?

Part 5: EOS

Part 6: Hyperledger Indy

Part 7: Consensus

  • What are the advantages and disadvantages of the E-O-V consensus process in Hyperledger Fabric?

Part 8: Software

  • Tell us about three design patterns you usually use. Write the pseudo-code Implementation of Observer Pattern
  • Implement pseudo-code to distribute work among multiple threads and wait for them to finish
  • Give me three examples of how to waste space (memory) to improve performance
  • What is padding, packing in memory alignment?

Part 9: Java

  • Explain Java’s method argument passing method. What’s Shallow Copy / Deep Copy.
  • What is the logic error of the following servlet call code (target is exected once after passing through filters)?
//// Filter 
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {

        ...  인코딩처리 OR
        ...  로깅처리 OR
        ...  인증처리

        chain.doFilter(request, response);
     
        ...  
}

//// FilterChain

public class FilterChain { 
   private List filters = new ArrayList(); 
   private Target target; 
   
   int currentFilter = 0; 

   public void addFilter(Filter filter){ 
      filters.add(filter); 
   } 

   public Filter getNextFilter(){ 
      if(currentFilter < filters.size()){ 
           return filters.get(currentFilter++); 
      } 
      return null; 
   } 
   public void doFilter(String request, String response){ 
         Filter f = getNextFillter(); 
         if(f != null){  
           f.doFilter(request,response,this);         
         } 
           
         target.execute(request,response); 
   } 

   public void setTarget(Target target){ 
      this.target = target; 
   } 
} 

Part 10: C++

  • What’s important about performance degradation and enhancements in C ++
  • Parse a single line to collect space-separated words, then code them to print out the word and the number of duplicates. (Performance and memory optimization
  • Write a function that takes a string as a parameter and returns a string with certain characters removed. (With performance optimization)
  • Briefly describe the auto / override / nullptr / constexpr / atomic keywords in C ++
  • Please explain the following code in C ++. (Consumers in the producer-consumer pattern, and there is only one consumer here)
Buffer BufferPool::get_buf(){   
   Buffer* buf = nullptr;
   std::unique_lock<std::mutex> ul(_mtx, std::defer_lock);

   while (buf == nullptr){
    ul.lock();
    if (_pool.empty()) _cond.wait(ul);

    if (!_pool.empty())  // 여기서 pool 이 empty 일 경우는?
    {
       buf = _pool.get();
    }
  }

   .... DO something ....
  return buf;
}

Part 11: Go

  • How is the select statement used in Go? Please explain the code below.
package main

import (
   "fmt"
   "time"
)

var scheduler chan string

func consuming (prompt string){
      fmt.Println("consuming 호출됨")
   select {
   case scheduler <- prompt:
      fmt.Println("이름을 입력받았습니다 : ", <- scheduler)
   case <-time.After(5 * time.Second):
      fmt.Println("시간이 지났습니다.")
   }
}

func producing (console chan string) {
   var name string
   fmt.Print("이름:")
   fmt.Scanln(&name)
   console <- name
}
func main() {
   console := make(chan string, 1)
   scheduler = make(chan string, 1)

   go func(){
      consuming(<-console)
   }()

   go producing(console)

   time.Sleep(100 * time.Second)
}

Part 12: Javascript

  • What are built-in Javascript objects / browser objects / HTML DOM objects
  • What is the difference between ajax and websocket communication
  • Show your previous works in React & CSS Styling

Part 13: Distributed systems

  • What is consistent hashing?
  • What is HAProxy?
  • What is Zookeeper and give two examples where you should use it

Part 14: Compilers

  • How does EOS charge for resources?
  • How to compute CPU, Memory and Storage usage in a program written in C ++ or Go?

Part 15: Cryptography

  • What is HMAC / PKI / ECDSA / ECDH
  • What is ECert in Hyperledger Fabric? Why does Hyperledger fabric use it?
  • How are zero knowledge proofs used in Fabric Identity Mixer?

Part 16: Database

  • Compare Red Black tree & B tree & Skip lists data structures.

Part 17: Messaging

Part 18: Networking / Socket

  • Tell me as much as you know the difference between socket communication between multithreaded / Select / Java NIO / ePoll / IOCP.

Experience with the following tools

  • Agile Management Techniques (* JIRA)
  • Product & Configuration Management (Bitbucket)
  • Containerization like Docker + Coobernate
  • Build Automation (* Bamboo)
  • Test Automation (* Unit Test gTest Study)
  • Issue Registration Automation (* JIRA)
  • Information sharing wiki management (confluence)
  • Information sharing chat management (slack)
  • Deployment Automation
  • Service Management Automation
  • Understanding Your Networking Infrastructure
  • Understanding Vertical / Horizontal Segmentation
  • Understanding and building a non-stop system (extending non-stop resources, etc.)
  • AWS Management

How to install nvm

This is a follow up to [How to install npm the right way]. It turns out that while convenient for Node development, nvm is notoriously slow. Thanks to reddit user sscotth we can solve that quite easily.

First, install nvm normally

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash

Then find the lines nvm added to your .rc file (bashrc or zshrc), delete that shit

# export NVM_DIR="$HOME/.nvm"
# [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
# [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

Next, add this to your .rc file

declare -a NODE_GLOBALS=(`find ~/.nvm/versions/node -maxdepth 3 -type l -wholename '*/bin/*' | xargs -n1 basename | sort | uniq`)

NODE_GLOBALS+=("node")
NODE_GLOBALS+=("nvm")

load_nvm () {
    export NVM_DIR=~/.nvm
    [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
}

for cmd in "${NODE_GLOBALS[@]}"; do
    eval "${cmd}(){ unset -f ${NODE_GLOBALS}; load_nvm; ${cmd} \[email protected] }"
done

Your globally installed programs like create-react-app will still use the current version of node, while it only loads once and not boggle down your terminal startup everytime.

Win-win