Tagged: ruby Toggle Comment Threads | Keyboard Shortcuts

  • kmitov 7:03 am on March 1, 2021 Permalink |
    Tags: , , ruby   

    JSON.parse(“1”) works. JSON.parse(“k1”) throws an exception 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    TL; DR; – JSON.parse(“1”) works. JSON.parse(“k1”) throws an exception

    This is a valid JSON


    This is NOT a valid JSON


    The code

    $ irb
    2.6.5 :001 > require 'json'
     => true 
    2.6.5 :002 > JSON.parse("1")
     => 1 
    2.6.5 :003 > JSON.parse("k1")
    Traceback (most recent call last):
            6: from /home/kireto/.rvm/rubies/ruby-2.6.5/bin/irb:23:in `<main>'
            5: from /home/kireto/.rvm/rubies/ruby-2.6.5/bin/irb:23:in `load'
            4: from /home/kireto/.rvm/rubies/ruby-2.6.5/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
            3: from (irb):3
            2: from /home/kireto/.rvm/gems/ruby-2.6.5/gems/json-2.5.1/lib/json/common.rb:216:in `parse'
            1: from /home/kireto/.rvm/gems/ruby-2.6.5/gems/json-2.5.1/lib/json/common.rb:216:in `parse'
    JSON::ParserError (809: unexpected token at 'k1')
    2.6.5 :004 > JSON.parse("\"1\"")

    It is always mistakes as simple as that, that are causing the bugs.


    I am sharing the details mostly for our team.

    When users use the Instructions Steps framework on the BuildIn3D and FLLCasts platforms we have a communication between the client and the server. The client sends information as JSON, the server parses it. For a few days some of the communication was missed. Not everything was recorded on the server. This was not anything critical, but about 360,000 events were not recorded and it shows how we did not understand JSON.parse entirely.

    The commit

    The commit to cause this was

    +        data = JSON.parse(events_array[0]["data"])
    +        referrer = data["referrer"] if data.is_a? Hash

    We get a string from events_array[0][“data”] and we parse this string. But this string is not always a JSON. It is sometimes just pure string. All the specs pass and we are committing the code.

    In the specs suite all the data that we are testing is “1”,”,2″- generally numbers. Why? Because this makes sense looking at the data of the spec and in the spec we’ve had only numbers as the string, but not a general string like “k1”.

    It bites. It bites like a shark.

    FabBRIX WWF, Shark in 3D building instructions
  • kmitov 10:20 am on February 4, 2021 Permalink |
    Tags: , , ruby   

    RSpec Matchers should be simple 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    Today on a review I saw an RSpec Matcher that was more complex than it should be. We’ve discussed it internally. I think this article could be useful to our team and to the community as a whole.

    An RSpec Matcher should be simple. It should return just ‘true’ or ‘false’

    IRL example of a complex RSpec Matcher

    This is the real live example of an RSpec Matcher from todays review

      RSpec::Matchers.define :have_content_picture_type_set_to do |picture_ref, content_picture_type, content_object|
        match do |page|
          content_object_undersore = content_object.model_name.to_s.underscore
          content_object_undersore = "tutorial" if content_object_undersore == "episode"
          content_object_undersore = "courses/#{content_object.course.to_param}/section" if content_object_undersore == "course_section"
          picture_ref_row = page.find(:xpath, "//tr[.//img[@src='#{picture_ref.content_picture.image_url}']]")
          js_agnostic_click_table_action picture_ref.to_param, "/author/#{content_object_undersore}s/#{content_object.to_param}/pictures/#{picture_ref.to_param}/edit", "Edit Picture"
          page.select content_picture_type, from: "content_picture_ref[content_picture_type]"
          click_on_and_expect "Update Picture", "Picture successfully updated"
          expect(page).to have_picture_ref_with_content_picture_type picture_ref, content_picture_type
        failure_message do |page|
          "expected page to have content_picture with the specified type, but was #{page.body}"

    There is a lot to unpack here.

    1. We do some calculations of paths with the whole content_object_underscore logic
    2. Then we find something on the page with the page.find method
    3. Then we call ‘js_agnostic_click_table_action‘ which is a complex method
    4. Then we select something from an HTML select
    5. Then we click a button with the click_on_and_expect
    6. At the end we expect something

    We should have only step 6. That’s the purpose of an RSpec Matcher. Return true or false if the page contains something or not.

    Why has this happened?

    Because RSpec.define :matcher is a function like any other ruby function. And as a ruby function you can do whatever you want with it. You can call anything you want in this function.

    Why should an RSpec Matcher be simple?

    That’s the convention for RSpec Matchers. Imagine the following spec:

        visit_pictures material
        expect(page).to have_picture_ref_with_content_picture_type material.content_picture_refs.first, "Thumbnail"

    With this spec you would expect visit a page and check if there is something on the page. But now have_pictuer_ref_with_content_picture_type makes calls, changes the db, modifies the current page and there is no way for you to understand it from reading the spec. Let alone reuse the Matcher in another situation.

    What feature is this about?

    The feature above is about showing a picture on the animated 3D models and building instructions that we show at BuildIn3D. Authors could choose which picture is the Thumbnail – example is this FabBrix Hen :).

    FabBRIX Farm Animals, Hen in 3D building instructions
  • kmitov 8:14 pm on January 31, 2021 Permalink |
    Tags: ruby   

    Ruby String#sub! and String#gsub! 

    (Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

    In Ruby String#sub! is different from String#gsub!. That much we know. But I am so used to .gsub that could not even see that the code in question uses .sub

    This one got me today. In the code we had

    self.text_content.sub!(/\( %7B%7B(%20)*/, "( {{ ")

    I went over to change the regex because of a new requirement and I deployed and published on production.

    But String#sub! replaces only the first occurrence of the regex, not all the occurrences. I could not event see that it was #sub! used and not #gsub!. Why would somebody one to replace only the first occurrence for this case – I don’t know.

  • kmitov 5:23 pm on May 21, 2020 Permalink |
    Tags: , , , , , ruby, ,   

    With assets in a rails engine it could be Gemfile vs gemspec dependency that is messing it up 

    You have a rails engine. It has an asset (like a scss file). You include this engine in another engine. How do you test that the assets is correctly resolved?

    There is a moment when you simply can not understand where and how the asset is resolved/not resolved. The solution actually is in what are the Gemfile and .gemspec for a rails engine. Both of them could describe dependencies, but they contain different things.

    The whole premise of the situation seems strange until you get to building a rails engine that would contain the “theme” of your app. We go in this situation with two of our platforms – FLLCasts.com and 3DAssemblyInstructions.com. We wanted to have different layouts in different gems that are providing different look an feel- separate the main layout from dashboard and from main. We also wanted to have this layouts in a different gems so that we could release them with different versions and test them as separate gems in separate builds.

    Let’s get into this step by step tutorial and by the end I am sure you would know much about how assets are resolved, packed and testes when they are within a rails. Again I am writing this tutorial to help spread the knowledge internally, but the problem is so common that it might be of interest to the community as a whole.

    Case 1 – simple app with an asset

    Just to get the understanding correctly and to have a base we would create a simple rails app with an asset and start it. I am using rails 6 and ruby 2.6.5

    $ rails new simple_assets_app
    $ cd simple_asset_app
    # Create a new asset.scss file with some content
    $ echo "asset_style { background-color: white}" > app/assets/stylesheets/asset.scss
    $ rails s
    => Booting Puma
    => Rails application starting in development 
    => Run `rails server --help` for more startup options
    Puma starting in single mode...
    * Version 4.3.5 (ruby 2.6.5-p114), codename: Mysterious Traveller
    * Min threads: 5, max threads: 5
    * Environment: development
    * Listening on tcp://
    * Listening on tcp://[::1]:3000
    Use Ctrl-C to stop

    In another terminal request the asset

    # Request the asset and dislay its content. Everything works. Perfect
    $ curl localhost:3000/assets/asset.scss
    asset_style { background-color: white}

    Case 2 – engine with an asset

    Now let’s build a rails plugin that would hold this asset

    $ rails plugin new simple_asset_engine --full
    $ cd simple_asset_engine/
    $ echo "asset_style { background-color: white}" > app/assets/stylesheets/asset.scss
    # You have to fix the TODOs in the gemspec. After you are ready do 
    $ rails s -p 3001
    => Booting WEBrick
    => Rails application starting in development http://localhost:3001
    => Run `rails server --help` for more startup options
    [2020-05-21 19:48:51] INFO  WEBrick 1.4.2
    [2020-05-21 19:48:51] INFO  ruby 2.6.5 (2019-10-01) [x86_64-linux]
    [2020-05-21 19:48:51] INFO  WEBrick::HTTPServer#start: pid=13394 port=3001

    Now that the port is different.

    Request the asset

    # Nothing strange here. Asset is delivered as expected.
    $ curl localhost:3001/assets/asset.scss
    asset_style { background-color: white}

    Conclusion – engine, no engine the asset is delivered.

    Case 3 – require simple_asset_engine in another engine

    This could be a requirement. It has happen to us. It is a valid case. It is rather strange to have one engine require another, but we are all grown ups here…

    $ rails plugin new host_engine --full
    # Fix todos in .gemspec
    # Add a dependency to simple_asset_engine. Asset should be found, rigth
    $ echo 'gem "simple_asset_engine", path: "~/axles/tmp/simple_asset_engine"' >> Gemfile
    # install gems - mainly the simple_asset_engine
    $ bundle install
    # Start server. Note the different port
    $ rails s -p 4010

    Request asset

    $ curl localhost:4010/assets/asset.scss
    asset_style { background-color: white}

    Asset is found.

    That’s the power of rails. It just works. You can have an asset in an app, asset in an engine, asset in an engine, required in another engine and the asset is always found. Except…

    This might sound as a conclusion, because everything works. Until you get to production, which is actually the only time it matters if anything works, but that’s another story.

    Our current host_engine depends on a gem with a local path in the local filesystem.

    host_engine: $ cat Gemfile
    source 'https://rubygems.org'
    git_source(:github) { |repo| "https://github.com/#{repo}.git" }
    # Declare your gem's dependencies in host_engine.gemspec.
    # Bundler will treat runtime dependencies like base dependencies, and
    # development dependencies will be added by default to the :development group.
    # Declare any dependencies that are still in development here instead of in
    # your gemspec. These might include edge Rails or gems from your path or
    # Git. Remember to move these dependencies to your gemspec before releasing
    # your gem to rubygems.org.
    # To use a debugger
    # gem 'byebug', group: [:development, :test]
    gem "simple_asset_engine", path: "~/axles/tmp/simple_asset_engine"

    This is something that we can not ship to production. That’s why we pack “simple_asset_engine” as a gem and place this gem in a gemrepo. This is not part of the tutorial so I will simulate it.

    We should also modify the host_engine.gemspec to depend on simple_asset_engine. We add a dependency.

    # in host_engine.gemspec we add
    spec.add_dependency "simple_asset_engine"

    This means – ‘add a dependency to “simple_asset_engine”‘. Our gem will host_engine will depend on ‘simple_asset_engine’, but and here is a but, but only for ‘production’ environment. So this means – “production”. To simulate the fact that we don’t have this gem in a repo we would change the dependency in the Gemfile to be only for production.

    # This means the same as spec.add_dependency 'simple_asset_engine' It add the gem as a dependency to production. We are doing it like this to simulate that we have 'spec.add_dependency' and the gem is coming from a gems repo
    gem "simple_asset_engine", group: [:production], path: "~/axles/tmp/simple_asset_engine"

    Now as you do request the asset a strange error occurs:

    $ curl localhost:4010/assets/asset.scss
    <!DOCTYPE html>
    <html lang="en">
      <meta charset="utf-8" />
      <title>Action Controller: Exception caught</title>
       <div id="Framework-Trace-0" style="display: none;">
          <code style="font-size: 11px;">
              <a class="trace-frames trace-frames-0" data-exception-object-id="47236752082100" data-frame-id="0" href="#">
                actionpack ( lib/action_dispatch/middleware/debug_exceptions.rb:36:in `call'
              <a class="trace-frames trace-frames-0" data-exception-object-id="47236752082100" data-frame-id="1" href="#">
                actionpack ( lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
              <a class="trace-frames trace-frames-0" data-exception-object-id="47236752082100" data-frame-id="2" href="#">
                railties ( lib/rails/rack/logger.rb:37:in `call_app'

    In the log you see

    ActionController::RoutingError (No route matches [GET] "/assets/asset.scss"):
    actionpack ( lib/action_dispatch/middleware/debug_exceptions.rb:36:in `call'
    actionpack ( lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
    railties ( lib/rails/rack/logger.rb:37:in `call_app'
    railties ( lib/rails/rack/logger.rb:26:in `block in call'
    activesupport ( lib/active_support/tagged_logging.rb:80:in `block in tagged'
    activesupport ( lib/active_support/tagged_logging.rb:28:in `tagged'
    activesupport ( lib/active_support/tagged_logging.rb:80:in `tagged'
    railties ( lib/rails/rack/logger.rb:26:in `call'
    sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:11:in `block in call'
    activesupport ( lib/active_support/logger_silence.rb:36:in `silence'
    activesupport ( lib/active_support/logger.rb:64:in `block (3 levels) in broadcast'
    activesupport ( lib/active_support/logger_silence.rb:36:in `silence'
    activesupport ( lib/active_support/logger.rb:62:in `block (2 levels) in broadcast'
    sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:11:in `call'
    actionpack ( lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
    actionpack ( lib/action_dispatch/middleware/request_id.rb:27:in `call'
    rack (2.2.2) lib/rack/method_override.rb:24:in `call'
    rack (2.2.2) lib/rack/runtime.rb:22:in `call'
    activesupport ( lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
    actionpack ( lib/action_dispatch/middleware/executor.rb:14:in `call'
    actionpack ( lib/action_dispatch/middleware/static.rb:126:in `call'
    rack (2.2.2) lib/rack/sendfile.rb:110:in `call'
    actionpack ( lib/action_dispatch/middleware/host_authorization.rb:82:in `call'
    railties ( lib/rails/engine.rb:527:in `call'
    rack (2.2.2) lib/rack/handler/webrick.rb:95:in `service'

    It is basically telling us that an error occurred. An error also should occur. It is logical, it is the right thing to do, but it can take hours to debug and understand this.

    The error occurs because we are starting the rails server in ‘development’ environment and the dependency to ‘simple_asset_engine’ is only for production.

    Case 3.1 – change to production and development

    # Change host_engine/Gemfile to have
    gem "simple_asset_engine", group: [:production, :development], path: "~/axles/tmp/simple_asset_engine"

    Restart server on port 4010.

    host_engine: $ rails s -p 4010
    => Booting WEBrick
    => Rails application starting in development http://localhost:4010
    => Run `rails server --help` for more startup options
    [2020-05-21 20:14:55] INFO  WEBrick 1.4.2
    [2020-05-21 20:14:55] INFO  ruby 2.6.5 (2019-10-01) [x86_64-linux]
    [2020-05-21 20:14:55] INFO  WEBrick::HTTPServer#start: pid=14167 port=4010

    Request the asset

    $ curl localhost:4010/assets/asset.scss
    asset_style { background-color: white}

    Now it is time for conclusion

    spec.add_dependency in a gemspec gives you a dependency for production. There is a ‘spec.add_development_dependency’ that exists, but there are great discussion about it here https://github.com/rubygems/rubygems/issues/1104 Read more there. Really. Read more.

    But as we are trying to test assets separately in an engine it is important to understand what Gemfile and gemspec could be used for. If an asset is not found from a dependency of a gemspec, probably the whole dependency is required for a different environment. In the same time if the dependency contains only assets, it might be difficult to get what is happening at first sight and is it your fault, or that the F..k is sprockets doing or event what the bigger f..k is webpacker doing. But for this case it is just plain old ruby dependencies that are messing us up.

  • kmitov 9:56 am on April 4, 2020 Permalink |
    Tags: , , geminabox, rake, ruby   

    bundle exec vs non bundle exec. 

    This article is part of the series [Everyday code].

    We use bundler to pack parts of the Instruction Steps Framework, especially the parts that should be easy to port to the rails world. We learned something about Bundler so I decide to share it with everybody.

    TL; DR;

    Question is – which of these two should you use:

    # Call bundle exec before rake
    $ bundle exec rake 
    # Call just rake
    $ rake

    ‘bundle exec rake’ will look at what is written in your .gemspec/Gemfile while rake will use whatever is in your env.

    Gem.. but in a box
    gem inabox with bundler

    Bundle exec

    For example we use geminabox, a great tool to keep an internal repo of plugins. In this way rails projects could include the Instruction Steps framework directly as a gem. This makes it very easy for rails projects to use the Instruction Steps.

    To put a gem in the repo one must execute:

    $ gem inabox

    You could make this call in three different ways. The difference is subtle, but important.

    Most of the time the env in which your shell is working will be almost the same as the env in which the gem is bundled. Except with the cases when it is not.

    From the shell

    # This will use the env of the shell. Whatever you have in the shell.
    $ gem inabox

    From a rake file

    If you have this rake file

    require 'rails/all'
    task :inabox do 
      system("gem inabox")

    then you could call rake in the following ways:

    rake inabox

    # This will call rake in the env defined by the shell in which you are executing
    $ rake inabox

    bundle exec rake inabox

    # This will call rake in the env of the gem
    $ bundle exec rake inabox

    When using the second call bundle will look at the ‘.gemspec’/’Gemfile’ and what is in the gemspec. If non of the gems in the .gemspec adds the ‘inabox’ command to the env then the command is not found and an error occurs like:

    ERROR:  While executing gem ... (Gem::CommandLineError)
        Unknown command inabox

    If ‘gem inabox’ is called directly from the shell it works, but to call gem inabox from a rake job you must have ‘geminabox’ as development dependency of the gem. When calling ‘gem inabox’ from a shell we are not using the development env of the gem, we are using the env of the shell. But once we call ‘bundle exec rake inabox’ and it calls ‘gem inabox’, this second call is in the environment of the gem. So we should have a development_dependency to the ‘geminabox’ gem:

     spec.add_development_dependency 'geminabox'

    Nice. Simple, nice, logical. One just has to know it.

compose new post
next post/next comment
previous post/previous comment
show/hide comments
go to top
go to login
show/hide help
shift + esc