Kiril Mitov blog

From a ticket to deploy in an hour. jQuery runs the scripts, DOM does not.

(Everyday Code – instead of keeping our knowledge in a README.md let’s share it with the internet)

This is an article of a production incidents with Rails/Stimulus/Rails-ujs/DOM and jQuery.

We made mistake that made us lose 31 registrations on the FLLCasts and a few more on the BuildIn3D platform. The stack involved includes Ruby on Rails with Rails-ujs, Stimulus, DOM and jQuery. I will enter into some details about how it occurred, why and how we resolved it in under an hour. I am writing this article mainly to share with my our team, but I am sure it could be useful for other teams.

The incident – people could not Sign Up on the platforms

On the Sign up page there is a captach. Years ago we found our that the captcha helps us reduce invalid registartion.
There is also one more thing on this platform – usernames. As the FLLCasts platforms is used by students and academies, one of the very useful features on the platform is the automatically generated username. Users type their names in the “Full name” field and a username is automatically generated for them. In this way there is a unique username for users because email is not unique – there could be many users with the same email.

To generate a new username we send a request to the server and refresh form. We use Stimulus and rails-ujs for this.

The is an “change->registrations#refreshForm” for the input field for the name. When the full name is changed the form is refreshed and a unique username is returned from the server.

<%= user_form.fields_for :data, {parent_builder: user_form} do |data_form| %>
  <%= data_form.text_field :name_attribute,
    {
      icon_prepend: "icon-finance-067 u-line-icon-pro",
      autofocus: true,
      data: {
        registrations_target: "name",
        action: "change->registrations#refreshForm"
      },
      autocomplete: "off"
    }
  %>

  <%= data_form.email_field :email %>
<% end %>

Here is how it should work:

The incident was that when the form was refresh the captcha got lost and we were not showing it after that. This means that on the server we were checking the captcha, but there was not captcha on the client.

The ticket

A wild ticket appears from a user.

Hi,

We wanted to enroll our Kid for Personal B programme and we were trying to Sign Up to pay online , but we are unable to sign up using our details.
We get an error message though we tried a few times.

Please help to sort this out.

Thanks

Regards

This is a really wild ticket. The user wants to buy and they can not sign up. 1,2,3 go.

The problem

We’ve made the following commit in the registration form

--- a/app/views/devise/registrations/new.js.erb
+++ b/app/views/devise/registrations/new.js.erb
@@ -1 +1 @@
-$("#<%= @form_id %>").replaceWith("<%= escape_javascript(render 'form') %>")
\ No newline at end of file
+document.getElementById("<%=form_id%>").outerHTML= '<%= escape_javascript(render "#{partial_name}") %>';
\ No newline at end of file

We are in the process of removing jQuery from the code. I saw this jQueyr call and decided to change it to a simple DOM call. It should be the same, right? No…

jQuery executes scrips while document.outerHTML does not

I knew this. But I did not consider it in this commit. What happens is that we receive the new form from the server and replace the form on the page with the new form received from the server that contains the generate username.

But in this form (and this form only on the whole platform) there is a recaptach. This recaptcha is a JavaScript and when dynamically changing the DOM the JavaScript must be executed. Well, jQuery automatically executed this for us. document.outerHTML does not.

Developing and RSpec system spec.

First I deployed the fix to production. It took about 5 minutes.

After that I developed the spec. This is how the spec looks like:

scenario "shows recaptach when dynamically reloading the form", js: true  do
    with_recaptcha_enabled do
      visit_sign_up

      expect(page).to have_recaptcha

      name_field.set unique_name
      email_field.set "#{SecureRandom.hex(16)}@example.com"

      # This means we have reloaded with js because we've set the email field 
      # and this has change the focus and there was a fire event for the name_field.
      #
      # We are now waiting for the username to appear on the screen
      expect(page).to have_username_filled

      expect(page).to have_recaptcha
    end
  end

We expect that there is a recaptcha before and after refreshing the form.

What surprised us

Looking at the data in the time frame this commit was on production – we’ve received 9 registrations on the platform. In the same previous period we’ve receive 40 registrations.

This means that we’ve lost 31 registrations on the platform.

What surprised us is that 9 people managed to register. The only way you could register on the platform during this period is to first enter your username. If you enter the username on your own we do not refresh the form as we don’t have to generate a username for you. This means that about 25% of the people start with their username and about 75% of the users at FLLCasts start the registration with their name or email.

Good to know. Good to know.

Conclusion

Looking at the issue we could have prevented it in a number of ways. In a test environment it is difficult to have a recaptcha because there is no way to test that the recaptcha works. After all that is the whole purpose of a recaptcha – to prevent bots and a Selenium driver is exactly a bot. But it turned out we’ve missed it as a scenario in the specs. It is always a missing spec.