Tagged: continuous integration Toggle Comment Threads | Keyboard Shortcuts

  • kmitov 10:45 am on April 3, 2020 Permalink |
    Tags: bash, continuous integration,   

    99 versions are not good enough – [Everyday Code] 

    This article is part of the series [Everyday Code]

    You’ve done nothing until you release more than 99 versions of your product. 99 versions are just not good enough.

    TL;DR;

    Today we released version 103 of is-core – the core of the Instruction Steps Framework. We noticed a bug. Generally the build would produce two files:

    is-core-sdk-6.0.0.pre.103.js - that is the current version 
    is-core-sdk-latest.js  - this is pointing to the content of the latest version. 

    Problem was that while the current version was 103, the latest version in is-core-sdk-latest.js was pointing to version is-core-sdk-6.0.0.pre.99.js.

    As a conclusion – You have done nothing until you’ve released at least 100 versions of your software (and probably at least it works through a millennium shift with a leap year, but that’s another story)

    Details

    It’s pretty simple actually. This is what we were doing to get the latest file generated:

    # Creates is-core-sdk-latest.js link to the latest compiled 
     cd ../../release
     rm is-core-sdk-latest.js -f
    -latest=`find is-core-sdk-* -type f | tail -1`
    +latest=`ls -1v is-core-sdk* | tail -1`
     echo "Latest sdk is: $latest"
    

    Notice the find is-core-sdk-* -type f | tail -1 If the files are like

    # Find all the files but they are listed in non natural order of the integer for the version.
    # This code is: BAD
    $ find is-core-sdk-* -type f 
    is-core-sdk-6.0.0.pre.102.js
    is-core-sdk-6.0.0.pre.103.js
    is-core-sdk-6.0.0.pre.97.js
    is-core-sdk-6.0.0.pre.98.js
    
    # If we get just the tail it will give us version 99 which is clearly not right
    $ find is-core-sdk-* -type f | tail -1
    is-core-sdk-6.0.0.pre.99.js
    

    I have done this mistake at least a few times in my career.

    Solution is an option in ls:

    # This code is GOOD
    # This will list all the files
    $ ls -1v is-core-sdk*
    is-core-sdk-6.0.0.pre.97.js
    is-core-sdk-6.0.0.pre.98.js
    is-core-sdk-6.0.0.pre.102.js
    is-core-sdk-6.0.0.pre.103.js
    
    # This will get just the last
    $ ls -1v is-core-sdk* | tail -1
    is-core-sdk-6.0.0.pre.103.js
    

    Moral of the story

    For months I thougth we have a rock solid infrastructure. There was almost no failed build. Delivery to production is in 2 minutes for a pretty complex framework with a lot of projects and modules. And then it “broke” after months of stable work just as we were to release version 100.

    Show me your 100-th version of your product. Then we can talk.

     
  • kmitov 6:30 pm on March 28, 2020 Permalink |
    Tags: continuous integration, git   

    Time to be praised as “The Guru” in your office – or how to move a folder from one Git repo to another Git repo and preserve the history 

    Ok. This is the 5th or 6th time that I am doing this and one of my colleagues asked me.

    Could you please write down how you do it so that we could do it without you next time.

    So here it is. 2 minutes read and you can move files from one git repo to another repo and move the commit history for some of the files. Commands below, but first:

    Word of caution (please):

    Such an enormous power under your fingers would make you and object of great attention in your company. Colleagues will sing songs about you. You would be praised as “The Guru”. Small talks before meetings will start with “Have you heard of that dude that can move files to a new git repo and keep all the history”. (believe me, it is like walking on water). Also I can not promise, but I am pretty sure you can get laid with such knowledge. I once got laid for knowing JUnit and Eclipse so… who knows.

    The disadvantage is that once people learn about you and your knowledge, you will be their go to person for questions about git. Git is quite complex and many people are lazy when it comes to reading documentation, so naturally many people would start asking you questions. Mostly stupid questions, of course. Can you handle the load?

    Task

    Moving folder integration/processors from old_repo repo to new_repo

    Commands

    # Enter new repo
    $ cd new_repo/
    
    # Make sure you are up to date
    $ git pull
    Already up to date.
    
    # Check remotes. Just to see what you've got
    $ git remote -v
    origin  git@host:new_repo (fetch)
    origin  git@host:new_repo (push)
    
    # You are in the new repo. Add the remote to the old_repo
    $ git remote add old_repo git@host:old_repo
    
    # Make sure the remote for old_repo is added
    $ git remote -v
    old_repo  git@host:old_repo (fetch)
    old_repo  git@host:old_repo (push)
    origin  git@host:new_repo (fetch)
    origin  git@host:new_repo (push)
    
    # Fetch from the branch of old_repo as it is now an origin
    $ git fetch old_repo
    ...
    From host:old_repo
     * [new branch]      dev                     -> old_repo/dev
    
    # Checkout the branch from old_repo
    $ git checkout --track old_repo/dev
    
    # Remove all the paths that you don't need. Keep the paths that you do need. Bunch of magic. Better read the documentation about it.  
    $ git filter-branch --force --index-filter   "git rm --cached -r --ignore-unmatch PATH_1 PATH_2 EVERYPATH_THAT_DOES_NOT_INCLUDE_INTEGRATION/PROCESSORS"   --prune-empty --tag-name-filter cat -- --all
    
    # Return to your master branch
    $ git checkout master
    
    # Merge the already filtered branch to your master.
    $ git merge dev  --allow-unrelated-histories
    
    # Think not twice, but three times. After this there is not turning back. It's the Fame or the Shame!!!
    $ git push -f

     
  • kmitov 6:26 am on March 23, 2020 Permalink |
    Tags: continuous integration, prettier   

    Simple warning goes a long way 

    TL;DR;

    Just warn people with a simple message when you are deprecating a behavior in your tool and you are introducing a breaking change. It’s not that difficult

    Story

    Yesterday I kind of wake up to a nasty surprise in our local Continuous Integration.

    Continuous Integration on Jenkins failing miserably

    The problem

    Prettier (https://github.com/prettier/prettier/) have released a breaking change from version 1.19.1 to 2.0.1. This breaks most of our projects.

    The bigger problem

    Prettier is one of the nicest tools we’ve used. It allows us to keep the code formatted. It is also integrated in our CI and if a file is not properly formatted when committed the build fails.

    Several months ago it took us 17.5 hours to integrate Prettier to all developers and all projects and since then we had no problems.

    Then the update happened.

    I have nothing against breaking changes in an API or a project. I welcome them especially in non-critical tools as formatting the code. People learn. People need to learn and building and maintaining an API takes practice, consideration and a lot of experience. I have personally broken some API(s) that I’ve developed in the past. But what I think about breaking changes is that you should properly communicate this with you clients. We are using prettier in a very simple way. Here is the command:

    npx prettier app/**/*.js test/dummy/spec/javascripts/**/*_spec.js vendor/assets/javascripts/gcc/externs/*.externs.js --write --config prettier_conf.json"  

    That’s it. Turns out that as of version 2.0.1 prettier have broken this behavior and now if the project has no files for any of the globs it will return an error.

    For version 2.0.1

    $ mkdir pretti
    $ cd pretti/
    /pretti$ touch some.js
    /pretti$ npx prettier --version
    2.0.1
    /pretti$ ls
    some.js
    /pretti$ npx prettier app/**/*.js *.js
    [error] No files matching the pattern were found: "app/**/*.js".
    /pretti$ echo $?
    127

    For version 1.19.1

    $ mkdir pretti
    $ cd pretti/
    /pretti$ touch some.js
    /pretti$ npx prettier --version
    1.19.1
    /pretti$ ls
    some.js
    /pretti$ npx prettier app/**/*.js *.js
    /pretti$ echo $?
    0

    See what they did there. Previously if a pattern was not matched prettier returned 0 and now it returns 127, which for a Linux is just error.

    Conclusion and solution

    “Professionals have standards”

    When designing tools have Interoperability in mind. Do breaking changes, but release a version that warns people for the deprecation and for the breaking change they are about to experience. Like a simple print to console in version 1.99 (the one before the breaking change) that says “hey, this is deprecated and will be removed in 2.0. Please read here ‘link’ so that your clients don’t break, you don’t open issues on our github and you don’t write blog posts. Stay safe.”


     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel