Tagged: buildin3d Toggle Comment Threads | Keyboard Shortcuts

  • kmitov 9:01 am on April 6, 2022 Permalink |
    Tags: buildin3d, , ui, ux   

    How a bad UI message prevented us from resolving a 5 days feature downtime 

    This article is about a UI message in our platform that was confusing enough that it prevented us from resolving a 5 days downtime on a feature. We could have resolved it much faster, but because the message was confusing it made things worse. I hope that UI experts and engineers (like us) could benefit and get an insight of how some things are confusing to the users.

    Context

    Our https://www.buildin3d.com service for visualizing animated and interactive 3D assembly instructions is used by one of the well known robotics education platforms https://www.fllcasts.com. At FLLCasts there are instructions of LEGO robots and mission models from the FIRST LEGO League robotics competitions. Here is an example instruction – https://www.fllcasts.com/materials/1393-bag-12-helicopter-with-food-package-first-lego-league-2021-2022-cargo-connect

    On the left you see the message “Downloading initial steps…’

    Problem

    5 days ago a user wrote a ticket to us with a question:

    “Why can’t I download the instructions?”.


    I naturally responded to the user “Well, you can’t because this is how we’ve built them to work. They are open in the browser and they can not be downloaded. What are you trying to do anyway?” (oh, my ignorance)

    The user replied in the ticket

    “I just want to build them.”

    Obviously there was a great miscommunication between us. The user was trying to build the instructions, but was using the term “Download”. Why? Well, because we taught them so.

    5 days later I found out the following thing. Some of the instructions were not working. When users were visiting them we, as “ignorant engineers” were showing a message that said “Downloading initial steps…” while the instruction was loading. In our engineering mind we were really downloading the instruction to the browser. Some of the steps of the instruction were downloaded from the server to the client, which is the browser.

    When the user got in touch with us he said “Hey, why can’t I download the instructions”.
    When asked this question I assumed in the ticket that the user wanted to download the instructions on their file system and be able to open them from there in an offline mode. Which is generally what “Download” means. I made a really wrong assumption of what the user was trying to do. He was trying to load and view the instructions. He was not trying to “Download” them. But we were telling them that we are “Downloading” the instructions and they naturally used this term.

    The implications

    For 5 days about 5-6% of our instructions were not working correctly. We could have resolved this the right way if I paid a little more attention to what the user was asking in the ticket or if the message was “Loading initial steps…” instead of “Downloading initial steps…”

    You learn something every day.

     
  • kmitov 7:03 am on March 1, 2021 Permalink |
    Tags: buildin3d, ,   

    JSON.parse(“1”) works. JSON.parse(“k1”) throws an exception 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    TL; DR; – JSON.parse(“1”) works. JSON.parse(“k1”) throws an exception

    This is a valid JSON

    "1"

    This is NOT a valid JSON

    "k1"

    The code

    $ irb
    2.6.5 :001 > require 'json'
     => true 
    2.6.5 :002 > JSON.parse("1")
     => 1 
    2.6.5 :003 > JSON.parse("k1")
    Traceback (most recent call last):
            6: from /home/kireto/.rvm/rubies/ruby-2.6.5/bin/irb:23:in `<main>'
            5: from /home/kireto/.rvm/rubies/ruby-2.6.5/bin/irb:23:in `load'
            4: from /home/kireto/.rvm/rubies/ruby-2.6.5/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
            3: from (irb):3
            2: from /home/kireto/.rvm/gems/ruby-2.6.5/gems/json-2.5.1/lib/json/common.rb:216:in `parse'
            1: from /home/kireto/.rvm/gems/ruby-2.6.5/gems/json-2.5.1/lib/json/common.rb:216:in `parse'
    JSON::ParserError (809: unexpected token at 'k1')
    2.6.5 :004 > JSON.parse("\"1\"")
    

    It is always mistakes as simple as that, that are causing the bugs.

    Details

    I am sharing the details mostly for our team.

    When users use the Instructions Steps framework on the BuildIn3D and FLLCasts platforms we have a communication between the client and the server. The client sends information as JSON, the server parses it. For a few days some of the communication was missed. Not everything was recorded on the server. This was not anything critical, but about 360,000 events were not recorded and it shows how we did not understand JSON.parse entirely.

    The commit

    The commit to cause this was

           else
    +        data = JSON.parse(events_array[0]["data"])
    +        referrer = data["referrer"] if data.is_a? Hash
    +
    

    We get a string from events_array[0][“data”] and we parse this string. But this string is not always a JSON. It is sometimes just pure string. All the specs pass and we are committing the code.

    In the specs suite all the data that we are testing is “1”,”,2″- generally numbers. Why? Because this makes sense looking at the data of the spec and in the spec we’ve had only numbers as the string, but not a general string like “k1”.

    It bites. It bites like a shark.

    FabBRIX WWF, Shark in 3D building instructions
     
  • kmitov 7:48 am on February 3, 2021 Permalink |
    Tags: , buildin3d, python, , ,   

    Where is the redundancy? 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    I think calling a method thrее times is redundant. But then again, you have to balance. Today’s article is about a code review that at the end took like a few hours in total of different discussions and I believe it is important. This kind of things take time. Failed builds, difficult specs.

    The IRL example – Where is the redundancy? Is this DRY?

     @dataclass(frozen=True)
     class LdrawLine(abc.ABC):
    +    default_x: ClassVar[float] = 23
    +    default_y: ClassVar[float] = 45
    +    default_z: ClassVar[float] = 0
         """
             This is the abstract class which every LdrawLine should implement.
         """
    @@ -85,11 +88,19 @@ class LdrawLine(abc.ABC):
             elif line_args[1] == "STEP":
                 line_param = _Step()
             elif line_args[1] == "ROTSTEP":
    -            rostep_params = {
    -                "rot_x": float(line_args[2]),
    -                "rot_y": float(line_args[3]),
    -                "rot_z": float(line_args[4])
    -            }
    +            if line_args[2] == "END":
    +                rostep_params = {
    +                    "rot_x": LdrawLine.default_x,
    +                    "rot_y": LdrawLine.default_y,
    +                    "rot_z": LdrawLine.default_z,
    +                    "type": "ABS"
    +                }
    +            else:
    +                rostep_params = {
    +                    "rot_x": float(line_args[2]),
    +                    "rot_y": float(line_args[3]),
    +                    "rot_z": float(line_args[4])
    +                }
     
    

    This piece of code (along with a few other changes in the commit) were the root of a 2 hours discussion in the team. A spec failed because some things were int while we were expecting them to be float.

    Calling ‘float’ three times like this is redundant

    The reason I think like this is that if you have to change and would like to have a double value or an int value you would have to change the code in three places.

    Probably e better solution would be:

    +            if line_args[2] == "END":
    +                rostep_params = {
    +                    "rot_x": LdrawLine.default_x,
    +                    "rot_y": LdrawLine.default_y,
    +                    "rot_z": LdrawLine.default_z,
    +                    "type": "ABS"
    +                }
    +            else:
    +                rostep_params = {
    +                    "rot_x": line_args[2],
    +                    "rot_y": line_args[3],
    +                    "rot_z": line_args[4]
    +                }
                      # We are adding a loop to call the float
    +                for key in ["rot_x", "rot_y", "rot_z"]:
    +                     rotstep_params[key] = float(rotstep_params[key])
     
    

    We call float only once at a single place. Now we have to deal with the fact that we have “rot_x” as a variable in two places, yes, that is true, but this could easily be extracted and we can iterate over the rostep_params values. But now we have consistency.

    Is it harder to read? Probably it is a little harder. Instead of simple statements you now have a loop. So you lose something, but you gain something. A float function that is called at a single place.

    What are we doing with these rotsteps?

    ROTSTEP is a command in the LDR format. We support LDR for 3D building instructions. Here is one example with a FabBrix Monster that uses the LDR ROTSTEP as a command:

    FabBRIX Monsters, Cthulhu in 3D building instructions
     
  • kmitov 8:31 pm on January 28, 2021 Permalink |
    Tags: buildin3d, , ,   

    How the software becomes unmaintainable? – a practical example 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    An ugly, sometimes over neglected truth of the industry is that the software that we work on does not become unmaintainable and tedious and difficult to work with overnight. It becomes like this with every change we introduce. While each individual change might not be “that bad” when they pile up we no longer have a well decoupled, maintainable system. We end up with a mess, which we than re-write with a new framework, a new team, new concepts and architecture just trying to do better. But the problem most of the time is in the single change that we do today – is it making the software system better or worse?

    This article is about a practical example of today and how we got to stop it.

    To better or worse

    When developing software incrementally we introduce changes. There are basically two options – the new changes would make the system better or they will make it worse.

    Here is the change from today

    --- a/file1.rb
    +++ b/file1.rb
    @@ -8,6 +8,7 @@ module IsBackend
     
         def embed
    +      @full_video_src = @material.video_refs.where(usage_type: "full_video").first.try(:video).try(:source_url)
     
    diff --git a/file2.rb b/file2.rb
    index bce7cd1a0..43b3768ea 100644
    --- a/file2.rb
    +++ b/file2.rb
     
       before_action do
         @namespaces = [:author]
    +    @full_video_src = @material.video_refs.where(usage_type: "full_video").first.try(:video).try(:source_url)
       end
    
    diff --git a/file3.rb b/file3.rb
    index 650456cb0..8c772288a 100644
    --- a/file3.rb
    +++ b/file3.rb
    @@ -41,6 +41,8 @@ class MaterialsController < CommonController
         @client_import_path = "shared" if Rails.application.config.platform.id == "b3"
    +    @preview_video_src = @material.video_refs.where(usage_type: "preview").first.try(:video).try(:source_url)
    +    @full_video_src = @material.video_refs.where(usage_type: "full_video").first.try(:video).try(:source_url)
       end
     

    The logic is not important. We basically get the video for a material. What’s important is the code is practically the same in all 3 places. 4 calls in three files and they are all the same.

    In the past I was leading a class in Software Development. One thing I tried to teach each student was:

    When you copy and paste you introduce a bug. That’s a fact of the industry.

    The reason is that once you copy/paste you would have to support the same logic in more than one place and you would simply forget about the second place the next time you would like to change the logic. It might not be you, it might be colleagues working years from now on the same code, but they will forget to change both places. That’s the problem with redundancy.

    Remove the redundancy, and you remove most of the bugs

    How did we stop it?

    Simple review. Just ask a second person to check your commit. This simple review process protects you from a large portion of the other bugs that are still left after you’ve cleared all the redundancies (of course).

    Enjoy

    The code in question is part of the logic delivering these instructions. See how the video is displayed. That’s what the feature was about. Enjoy.

    Torvi – Ball shooting Lego machine
     
  • kmitov 7:54 am on January 21, 2021 Permalink |
    Tags: buildin3d, , , , ,   

    How and why we test JavaScript – Jasmine and Teaspoon 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    We are serious about specs. A good suite of specs has helped us many times especially when it comes to regressions. For the JavaScript part of our stack many of our specs and using Jasmine and ran with Teaspoon in the Browser environment. This article is about how we use Teaspoon and Jasmine to run our specs for pure JavaScript frameworks like Instructions Steps, and BABYLON.js. Both BuildIn3D and FLLCasts are powered by this specs, but we use them mostly for the Instructions Steps Framework.

    Why JavaScript specs?

    In the Rails community specs are important. In the JavaScript community – well, they are not that well respected. We can debate the reasons, but this is our observation.

    When we implement specs for the platforms we like to use Rails system specs. They give us exactly what we need. Let’s take a real life example.

    Filtering by category

    We visit the page at BuildIn3D for all the 3D models & building instructions. We filter them by brand. We click on a brand and see only the 3D models & instructions for this specific brand.

    Here is the spec. Short and sweet.

    scenario "/instructions shows the instructions and can be filted by category", js: true do
      # go to the third page as the materials are surely not there
      visit "/instructions?page=3" 
      expect(page).not_to have_link material1.title
    
      click_category category1
    
      # We make sure the url is correct
      expect(page).to have_current_path(/\/instructions\?in_categories%5B%5D=#{category1.id}$/, url: true)
    
      # We make sure that only material1 is shown and material2 is not shown
      # The materials are filtered by category
      expect(page).to have_link material1.title
      expect(page).not_to have_link material2.title
    endT

    The spec is a Rails system spec. Other articles enter into more details about them. The point is:

    With a Rails system spec we don’t concern ourselves with JavaScript.

    We visit the page, we click, we see that something has changed on the page – like new materials were shown and others were hidden.

    What’s the use case for JavaScript specs then?

    Take a look at the following embedded BuildIn3D instruction. It is coming live and it has a next button. Click on the next button.

    TRS the Turning Radio Satellite construction from GeoSmart and in 3D

    Here is the actual spec in JavaScript with Jasmine and ran everyday with Teaspoon.

    it("right arrow button triggers TriedMoveIterator with 1", function(done) {
        // This is a JavaScript spec that is inside the browser. It has access to all the APIs 
        // of the browser. 
        const eventRight = new KeyboardEvent("keydown", { key: "ArrowRight" });
        document.onkeydown(eventRight);
        
        // Wait for a specific event to occur in the Instructions Steps (IS) framework
        // We are inside the browser here. There is no communication with the server
    
        IS.EventsUtil.WaitFor(() => this.iteratorListener.event).then(() => {
          expect(this.iteratorListener.event.getSteps()).toEqual(1);
          done();
        });
      });
    

    This the how we use JavaScript and this is why we need them:

    We need JavaScript specs that are run inside the browser to communicate with other JavaScript objects and the browser APIs for JavaScript apps.

    In this specific case there is this “iteratorListener” that monitors how the user follows the instructions. We need access to it. Otherwise it gets quite difficult to test. We can not write a spec to test what are the pixels on the screen that are drawn after clicking next. This will be … difficult to put it mildly. We don’t have to do it. We need to know that clicking the next button has triggered the proper action which will then draw the actual Geometry and Colors on the screen.

    How we use Jasmine and Teaspoon to run the JavaScript specs

    Years ago I found a tool called Teaspoon. Looking back this has been one of the most useful tools we’ve used in our stack. It allows us to have a Rails project (and we have a lot of those) and to run JavaScript specs in this Rails project. The best thing – it just works (the Rails way).

    # add teaspoon dependency
    $ cd test/dummy/
    $ rails generate teaspoon:install
    # write a few specs
    $ rails s -p 8889
    

    You start a server, visit the url in the browser and the specs are executed

    Tests are pure JavaScript and we use Jasmine.

    That’s it. Not much to add. It’s simple The specs are a simple JS file located in “spec/javascripts/” folder and here is an example for one of them

    describe("IS.EventDef", function() {
      describe("constructing throws error if", function() {
        it("event name is null", function() {
          expect(() => new IS.EventDef(null, {})).toThrow(new Error("Null argument passed."));
        });
    
        it("event conf is null", function() {
          expect(() => new IS.EventDef("smoe", null)).toThrow(new Error("Null argument passed."));
        });
    
        it("declaring an event, but the class attribute value is null", function() {
          expect(() => {
            const ext = IS.ExtensionDef.CreateByExtensionConf({
              extension: new IS.Extension(),
              events: {
                declaredEvent: {
                  class: null
                }
              }
            });
          }).toThrow("Declared event 'declaredEvent' class attribute is null!");
        });
      });
    });
    

     
  • kmitov 7:51 am on January 19, 2021 Permalink |
    Tags: buildin3d, , , , , ,   

    Same code – two platforms. With Rails. 

    (Everyday Code – instead of keeping our knowledge in an README.md let’s share it with the internet)

    We are running two platforms. FLLCasts and BuildIn3D. Both platforms are addressing entirely different problems to different sets of clients, but with the same code. FLLCasts is about eLearning, learning management and content management, while BuildIn3D is about eCommerce.

    What we are doing is running both platforms with the same code and this article is about how we do it. The main purpose is to give an overview for newcomers to our team, but I hope the community could benefit from it as a whole and I could get feedback and learn what others are doing.

    What do we mean by ‘same code’?

    FLLCasts has a route https://www.fllcasts.com/materials. Results are returned by the MaterialsController.

    BuildIn3D has a route https://platform.buildin3d.com/instrutions. Results are returned by the same MaterialsController.

    FLLCasts has things like Organizations, Groups, Courses, Episodes, Tasks which are for managing the eLearning part of the platform.

    BuildIn3D has none of these, but it has WebsiteEmbeds for the eCommerce stores to embed and put 3D building instructions and models on their E-commerce stores.

    We run the same code with small differences.

    Do we use branches?

    No, we don’t. Branches don’t work for this case. They are hard to maintain. We’ve tried to have an “fc_dev” branch and a “b3_dev” branch for the different platforms, but it gets difficult to maintain. You have to manually merge between the branches. It is true that Git has made merging quite easy, but still it is an “advanced” task and it is getting tedious when you have to do it a few times a day and to resolve conflicts almost every time.

    We use rails engines (gems)

    We are separating the platform in smaller rails engines.
    A common rails engine between FLLCasts and BuildIn3D is called fc-author_materials. It provides the functionality for and author to create a material both on FLLCasts and on BuildIn3D.

    The engine providing the functionality for Groups for the eLearning part of FLLCasts is called fc-groups. This engine is simply not installed on BuildIn3D, we install it only on FLLCasts.

    How does the Gemfile look like?

    Like this:

    install_if -> { !ENV.fetch('CAPABILITIES','').split(",").include?('--no-groups') } do
      gem 'fc-groups_enroll', path: 'gems/fc-groups_enroll'
      gem 'fc-groups', path: 'gems/fc-groups'
    end
    

    We call them “Capabilities”. By the default each platform is started with a “Capability” of having Groups. But we can disable them and tell the platform to start without Groups. When the platform starts the Groups are simply not there. We

    How about config/routes.rb?

    The fc-groups engine installs its own routes. This means that the main platform config/routes.rb is different from gems/fc-groups/config/routes.rb and the routes are installed only when the engine is installed.

    Another option is to have an if statement and to check for capabilities in the config/routes.rb. We still have to decide which is easier to maintain.

    Where do we keep the engines? Are they in a separate repo?

    We tried. We have a few of the engines in separate repos. With time we found out it is easier to keep them in the same repo.

    When the engines are in separate repos you have very strict dependencies between them. This proves to be useful but costs a lot in terms of development and creating a clear API between the engines. This could pay off when we would like to share the engines with the rest of the community like for example Refinery is doing. But, we are not there, yet. That’s why we found out we could spend the time more productively developing features instead of discussing which class goes where.

    With the case of all the rails engines in a single repo we have the mighty Monolith again, we have to be grown ups in the team and maintain it, but it is easier than having them in different repos.

    How do we configure the platforms?

    FLLCasts will send you emails from team [at] fllcasts [dot] com

    BuildIn3D will send you emails from team [at] buildin3d [dot] com

    Where is the configuration?

    The configuration is in the config/application.rb. The code looks exactly like this:

    platform = ENV.fetch('FC_PLATFORM', 'fc')
    if platform == 'fc'
      config.platform.sender = "team [at] fllcasts [dot] com"
    elsif platform == 'b3'
      config.platform.sender = "team [at] buildin3d [dot] com"
    

    When we run the platform we set and ENV variable called FC_PLATFORM. If the platform is “fc” this means “FLLCasts”. If the platform is “b3” this means “BuildIn3D”.

    In the config/environments/production.rb we are referring to Rails.application.config.platform.sender. In this way we have one production env for both platforms. We don’t have many production evns.

    Why not many production envs?

    We found out that if we have many production envs, we would also need many dev envs and many test envs and there will be a lot of duplication between them.

    That’s why we are putting the configuration in the application.rb. It’s about the application, not the environment.

    How do we deploy on heroku?

    First rule is – when you deploy one platform you also deploy the other platform. We do not allow different versions to be deployed. Both platforms are running the same code, always. Otherwise it gets difficult.

    When we deploy we do

    # In the same build we 
    # push the fllcasts app and then the buildin3d app
    git push fllcasts production3:master
    heroku run rake db:migrate --app fllcasts
    
    git push buildin3d production3:master
    heroku run rake db:migrate --app buildin3d

    In this way both platforms always share the same code, except for a short period of a few minutes between the deployment.

    How are the views separated?

    The platforms share a controller, but the views are different.

    The controller should return different views for the different platforms. Here is what the controller is doing.

    def show
       # If the platform is 'b3' return a different set of views
        if Rails.application.config.platform.id == "b3"
          render :template => Rails.application.config.platform.id+"/materials/show"
        else
          render :template => "/materials/show"
        end
      end

    In the same time we have the folders:

    # There are two separate folders for the views. One for B3 and one for FC 
    fc-author_materials/app/views/b3/materials/
    fc-author_materials/app/views/materials/

    How do we test?

    Testing proved to be challenging at first. Most of the time as the code is the same the specs should be the same right?

    Well, no. The code is the same, but the views are different. This means that the system specs are different. We write system and model specs and we don’t write views and controllers specs (if you are still writing views and controllers specs you should consider stopping. They were deprecated years ago).

    As the views are different the system specs are different.

    We tag the specs that are specifically for the BuildIn3D platform with a tag platform:b3

    
      context "platform is b3", platform: :b3 do
        before :each do
          expect_b3
        end

    When we run the specs we run first all the specs that are not specifically for b3 with

    $ rake spec SPEC="$specs_to_build" SPEC_OPTS="--tag ~platform:b3 --order random"

    Then we run a second suite for the tests that are specifically for the BuildIn3D platform.

    # Note that here we set an ENV and we use platform:b3 and not ~platform:b3
    $ FC_PLATFORM="b3" rake spec SPEC="$specs_to_build" SPEC_OPTS="--tag platform:b3 --order random"

    I see how this will become difficult to maintain if we start a third platform or a fourth platform, but we would figure it out when we get there. It is not something worth investing any resources into as we do not plan to start a new platform soon.

    Conclusion

    That’s how we run two platforms with the same code. Is it working? We have about 200 deployments so far in this way. We see that it is working.

    Is it difficult to understand? It is much easier than different branches and it is also much easier than having different repos.

    To summarize – we have a monolith app separated in small engines that are all in the same repo. When running a specific platform we install only the engines that we need. Controllers are the same, views could be different.

    I hope this was helpful and you can see a way to start a spin off of your current idea and to create a new business with the same code.

    There is a lot to be improved – it would be better to have each and every rails engine as a completely separate project in a different repo that we just include in the platform. But we still don’t have the requirement for this and it will require months of work on a 10 years old platform as ours. Once we see a clear path for it to pay off, we would probably do it in this way.

    For fun

    Thanks for stopping by and reading through this article. Have fun with this 3D model and building instructions.

    GeoShpere3 construction with GeoSmart set

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel