Tagged: tests Toggle Comment Threads | Keyboard Shortcuts

  • kmitov 7:13 am on November 10, 2021 Permalink |
    Tags: , , , , , tests   

    Migrating to jasmine 2.9.1 from 2.3.4 for teaspoon 

    We finally decided it is probably time to try to migrate to jasmine 2.9.1 from 2.3.4

    There is an error that started occurring randomly and before digging down and investigating it and and the end finding out that it is probably a result of a wrong version, we decided to try to get up to date with jasmine.

    Latest jasmine version is 3.X, but 2.9.1 is a huge step from 2.3.4

    We will try to migrate to 2.9.1 first. The issue is that the moment we migrated there is an error

    'beforeEach' should only be used in 'describe' function

    It took a couple of minutes, but what we found out is that fixtures are used in different ways.

    Here is the difference and what should be done.

    jasmine 2.3.4

    fixture.set could be in the beforeEach and in the describe

    // This works
    // fixture.set is in the describe
    describe("feature 1", function() {
      fixture.set(`<div id="the-div"></div>`);
      beforeEach(function() {
      })
    })
    // This works
    // fixture.set is in the beforeEach
    describe("feature 1", function() {
      beforeEach(function() {
        fixture.set(`<div id="the-div"></div>`);
      })
    })

    jasmine 2.9.1

    fixture.set could be only in the describe and not in the before beforeEach

    // This does not work as the fixture is in the beforeEach
    describe("feature 1", function() {
      beforeEach(function() {
        fixture.set(`<div id="the-div"></div>`);
      })
    })
    // This does work
    // fixture.set could be only in the describe
    describe("feature 1", function() {
      fixture.set(`<div id="the-div"></div>`);  
      beforeEach(function() {
        
      })
    })
     
  • kmitov 5:48 am on September 6, 2021 Permalink |
    Tags: , , , , tests   

    Refresh while waiting with RSpec+Capybara in a Rails project 

    This is some serious advanced stuff here. You should share it.

    A colleague, looking at the git logs

    I recently had to create a spec with Capybary+RSpec where I refresh the page and wait for a value to appear on this page. It this particular scenario there is no need for WebSockets or and JS. We just need to refresh the page.

    But how to we test it?

    # Expect that the new records page will show the correct value of the record
    # We must do this in a loop as we are constantly refreshing the page.
    # We need to stay here and refresh the page
    # 
    # Use the Tmeout.timeout to stop the execution after the default Capybara.default_max_wait_time
    Timeout.timeout(Capybara.default_max_wait_time) do
      loop do
        # Visit the page. If you visit the same page a second time
        # it will refresh the page.
        visit "/records"
        # The smart thing here is the wait: 0 param
        # By default find_all will wait for Capybara.default_max_wait_time as it is waiting for all JS methods 
        # to complete. But there is no JS to complete and we want to check the page as is, without waiting 
        # for any JS, because there is no JS. 
        # 
        # We pase a "wait: 0" which will check and return
        break if find_all(:xpath, "//a[@href='/records/#{record.to_param}' and text()='Continue']", wait: 0).any?
    
        # If we could not find our record we sleep for 0.25 seconds and try again.
        sleep 0.25
      end
    end

    I hope it is helpful.

    Want to keep it touch – find me on LinkedIn or Twitter.

     
  • kmitov 9:39 pm on November 28, 2020 Permalink |
    Tags: , , , tests,   

    Why we should never clear our DB before/after running specs. 

    One common “mistake” I’ve seen a couple of times is to clean the Database before/after specs are run. It seems to be a common practice with reasonable arguments. I think this is a bad idea. Here is why and what we should do instead.

    Why is the DB cleared before/after the specs

    When running specs that need access to a DB we might have to create a User or an Article or a Project model, then connect them in a certain way and test the business logic of our spec. After the spec is finished it is not wise to delete these objects from the DB directly in the spec. Sometimes it takes additional time, sometimes it executes additional logic. In most cases you don’t clear the DB after each and every spec.

    It is a good idea to clean the db before all the specs or after all the specs if they are successful. In this way we reset the DB only once, it saves some time and is much cleaner because you can plug in this behavior if you want to.

    Why the DB should not be cleared before/after the specs

    The simple answer is that our code will never, absolutely never work on a clean db in a production. If we have a test procedure that runs the specs against a clean and empty db they might pass when the db is clean. But what use do we have from code that could work in a clean environment, but could not work in a real production environment. The answer is – non.

    We don’t clean our db before/after each spec. In this way we’ve been able to track some really nasty bugs. Like slow queries that are slow only when you have too many users. Other cases involve special relations that are built in time. Like users that are part of an organization and the organization was once having one check for uniqueness of the user and now it has another check. Because the db is not cleared every time we make sure that it is properly migrated with all the needed migrations.

    We found out that a 7 years out test db that is not cleared is closer to a 7 years old production db.

    The test db is not the production db

    The test db is not the production db. It might have the same scheme, that is for sure, but the amount of data in them and the complexity of this data is different. What we need is code that could run on a production db. There is no use of any code that could run only in test environment.

    So here is what we do:

    We export the production db, we change some data like user emails, names and any other sensitive data and we import it as a test db. We run the specs on this db.

    In this way we make sure that the code could actually run on a real db before deploying it.

     
  • kmitov 6:48 am on August 12, 2019 Permalink |
    Tags: , , , tests   

    ‘if-else’ is Forbidden – why and how to reduce logic branching in your code. 

    Each time there is an if-else logic in your code there is some logic branching. A state is checked and based on the result from the check different path are taken. This means that two scenarios are implemented and two scenarios should be support from now on. This could quickly become a problem because the number of supported cases grows exponentially.

    The more decisions you make in the code the more cases you must support.

    At the end you start from Process1 but you have 4 different cases and path that you should support. These are 4 different tests that should be written and these are for different places where things could go wrong.

    The four different paths are Process 1,2,4; 1,2,5; 1,3,6; 1,3,7

    Demonstration of a live example and a real production bug

    We have Subscriptions in the platform. Each subscription has a max_users field that shows how many users you could add to this subscription. If max_users=1 this means you can have only one user using the subscription.

    When adding people to the subscription you have two cases

    1. You successfully add someone to the subscription and you show a message “Success”
    2. The max users is reached and you show a message “Error”

    The code in a simplified manner looks something like this:

    if subscription.save 
       form.show_message "Success"
    else 
       form.show_message "Error"

    While developing we’ve changed the code for the error from form.show_message “Error” to modal_dialog.show_message “Error”

    After that we’ve changed the implementation further and the code for modal_dialog.show_message “Error” was no longed called.

    As a result when the use tries to add someone to his subscription, for which they’ve payed, the app freezes and nothing happens. There is no error displayed and no user is added to the subscription.

    The bug occurred because with the latest changes we’ve forgot to manually check the second case of the if-else clause and there is was not test developed for this case.

    How to remove the if-else clause from this code

    subscription.save
    message = subscription.get_save_message
    form.show_message message

    The subscription.save knows what message to set based on whether the subscription was successfully saved. It could set Error or it could set Success. After that we just show whatever message the subscription has to the form. If it is empty that’s great. This means no errors have occurred. subscriptions.get_save_message could be implemented in many different ways. It could be on the subscription object or another object, but this depends on the technology and framework used. But after the method save is called the message is set and there is a single flow and now branches in our code. The method form.show_message is called a single time on a single place in our code. If we change the API of this method we would change in a single place and will not forget about the second place. There is always a single scenario. Save->Show message.




     
  • kmitov 7:21 am on January 8, 2019 Permalink |
    Tags: , chromedriver, , google-chrome, , , , tests   

    Chromedriver not filling all the whole password field in automated RSpec, Capybara, Feature Tests 

    This is such a lovely story. You will like it.

    When using google chromedriver to run capybara tests sometimes, just sometimes, especially if the tests are run in parallel, when the test has to fill a text field like a password, it fills only part of it. Last time checked for chromedriver 2.45

    TL; DR;

    Solution – google does not care, or it seems it is too difficult for the chromedriver team to resolve so there simply is no solution.

    What is the test?

    We are using Rails, Rspec, Capybara, Google chromedriver. We are developing feature tests. Tests are run in parallel with

    rake parallel:spec

    Here is the test in question. Simply fill a password on the form for upgrading a subscription, click on Confirm and expect to be redirected to a page that says – “You’ve upgraded your subscription”


    def submit_and_expect_success password
          # dialog opens to confirm with password
          fill_in "owner_password", with: password
          click_on "Confirm"
    
          expect_redirect_to "/subscriptions/#{subscription.to_param}"
    
          # If it redirects to this page, it means that the change was successful
          expect(page).to have_current_path "/subscriptions/#{subscription.to_param}"
    end

    And the tests are failing with timeout at expect_redirect_to. No, expect_redirect_to is a custom method, because we are using ActionCable to wait for subscription upgrade to finish. Because of the payment service at the back this sometimes takes a while and we want to show a nice progress and we need a websocket. But that being said the method is nothing special.

    module ExpectRedirect
      def expect_redirect_to url
        # If the path doesn't change before the timeout passes,
        # the test will fail, because there will be no redirect
    
        puts "expect url: #{url}"
        begin
          Timeout.timeout(Capybara.default_max_wait_time) do
            sleep(0.1) until url == URI(page.current_url).path
            page.current_url
          end
        rescue Timeout::Error=>e
          puts "Current url is still: #{page.current_url}"
          puts page.body
          raise e
        end
      end
    end

    If we are redirected to the url withing Capybara.default_max_wait_time than everything is fine. If not, we are raising the Timeout::Error.

    Parallel execution

    For some reason the test in question fails only when we are doing a parallel execution. Or at least mostly when we are doing parallel execution of the tests. So we moved through some nice articles to revise our understanding of Timeout again and again.

    https://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/

    But nevertheless the tests were failing with Timeout::Error on waiting for a redirect and in the html we could see the error returned by the server:

    <div><p>Invalid password</p></div>

    How come the password is Invalid

    No this took a while to debug and it seems rather mysterious but this is what we got:

    User password in DB is: 10124ddeaf1a69e3748e308508d916b6

    The server receives from the html form: 10124ddeaf1a69e3748e30850

    User password in DB is: 74c2a3e926420e1a30363423f121fc1e

    The server receives from the html from: 74c2a3e926420e1a3

    and so on and so on.

    Sometimes the difference is 8 symbols. Sometimes it is 2. Sometimes it is 16.

    It seems to be a client side issue

    Like. JavaScript. If there is an error this strange it has to be in the JavaScript. Right. There in the javascript we see:

    let form_data = form.serializeObject();
    this.perform('start_change', form_data);

    The form gets serialized. Probably it is not serialized correctly. Probably the values that we are sending are just not the values on the form. So I revised my knowledge on serializing objects in JavaScript with

    https://stackoverflow.com/questions/17488660/difference-between-serialize-and-serializeobject-jquery

    So far so good. But the serialization was not the problem. Here is what I did. I fixed all the passwords to be 32 symbols.

    let form_data = form.serializeObject();
     if(form_data["owner_password"].lenght != 32) {
            form_data["owner_password"] = "this was:" + form_data["owner_password"] + " which is less than 32 symbols"
          }
    this.perform('start_change', form_data);

    It it happened. The value of the password field was simply not 32 symbols long. It was not filled during the test.

    A little bit of search and we arrive at:

    https://github.com/teamcapybara/capybara/issues/1890

    and there in the bottom of the issue, there is the standard: “Not our problem resolution” with the link to:

    https://bugs.chromium.org/p/chromedriver/issues/detail?id=1771&q=sendkeys&sort=-id&colspec=ID%20Status%20Pri%20Owner%20Summary

    It seems that google chromedriver is not filling all the characters in the password field. It is doing it on random and is completely unpredictable.

    Issue still exists on:

    Issue still exist for
    
    Chrome: Version 71.0.3578.98 (Official Build) (64-bit)
    Chromedriver chromedriver --version
    ChromeDriver 2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5)
    Linux kireto-laptop3 4.4.0-141-generic #167-Ubuntu x86_64 GNU/Linux
    Description:	Ubuntu 16.04.5 LTS
    Release:	16.04
    Codename:	xenial

    Today we would try Firefox driver.


     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel