In this posts I have selected and compiled some Best Practices and Tips for Ruby users that I have learned up till now. They are the result of a year and a half of experience working on a Business Intelligence and Big Data project.

I believe the development of this functionality would have been more effective (faster and higher quality) if I had followed these tips from the beginning.

There are 30 tips. For the post not getting too long, I separated the list into two parts.

This is the first part, where I am more focus for optimization of tests (specs) and calls to the bank (queries).

In the second part, I talk about data treatment and its consistency. Tips and tricks when dealing with import and export ( CSV ) files.

Testing (specs)

1. Use FactoryGirl

The FactoryGirl has an excellent syntax and let you make cleaner and readable tests. In addition, it optimizes the time that it takes to create the specs, has centralized modeling code, is flexible, and enables advanced customization. That is, it is a hand in the wheel in cases of optimization of build time and more complex models.

2. Do not create Prefer build

It is good practice to use the same type of bench in production and test environments. This avoids surprises in production, even if the tests are passing.

One of the consequences of this, especially using ActiveRecord, is the time of the specs to be slower by having too many calls to the bench in the tests.

To reduce this impact, try to use it whenever you can.

3. Start with the exception

Many times we test only the happy, error-free path. The problem is that we leave the exceptions to the end, as background, without considering all the test scenarios.

My tip is to start with the exceptions, thinking about the most unlikely ways.

4. Describe the behavior

To understand how a model works, just read the description of the specs. But it’s not always so. It is quite common to see tests that do not describe the exact behavior of a model.

In the above example, everything is clearly described. I know that when I ask to drive a certain route and the car does not have enough fuel, it does not travel all the way you want. It stops halfway and the fuel runs out.

5. Test the functionality, not the implementation

If I change the logic of the method drive_forto this one below, the top test keeps going and the bottom one fails even with the correct logic.

It is very rare for anyone to worry about the functionality in creating the tests and / or analyzing this in the code review. Having to rewrite the tests every time you refactor or change the implementation is a waste of time.

6. rspec –profile

This option allows us to have instant feedback on spec time, making it easier to optimize the time for each test.

A good unit test should take less than 0.02 seconds. Functional ones usually take longer.

To not have to type in all the specs, you can insert the line –profilein the file .rspec, which is at the root of your project.

Bank calls (queries)

7. Use find_each, do noteach

With each, the memory usage increases along with the size of the base as it loads the entire base in one go.

Already find_each has the fixed consumption. It is influenced only by the size of the batch, which can be easily configured using the option batch_size.

In terms of implementation, nothing changes. Then use it find_eachat ease.

8. Use pluck, do notmap

9. Use select, not pluckwhen cascading

When you do this pluck, it executes the query and loads the entire list of results into memory. Then, another query is performed with the result obtained previously.

With this select, ActiveRecord stores only one Relation and joins the two queries into one.

This saves the memory that would be required to store the results of the first query. In addition, it eliminates overhead to establish a connection to the bank.

The databases have been evolving for a long time and if there is any optimization that it can do in this query, it will do better than ActiveRecord and Ruby.

10. Use exists?, do not any?

The runtime of the any grows according to the size of the base. This is because it makes one counting the table and then compares if the result is zero. Already exists puts one limit 1at the end and always takes the same time regardless of the size of the base.

11. Load only what you will use

We often iterate over the elements and use only a few fields. This ends up wasting time and memory because we have to load all the other fields that will not be used.

In the project, for example, I reduced more than 88% of memory usage. Reviewing and putting selecting all the queries, I was able to increase the competition with the same machine and the execution time was 12 times faster.

All this optimization gain in less than 1 hour of work. If this concern is already in your head at the time of creation, the additional cost is virtually zero.

Now we will focus on treatment and data consistency, tips and tricks when dealing with import and export ( CSV ) files.

Consistency of data

12. Set the time

Let’s say we have a routine that runs daily at midnight to generate a report from the previous day.

The first implementation may generate erroneous results if it is not run at that specified time.

In the second just ensure that it will run on the correct day and the result will be as expected.

In the third, it is possible to execute on any day, just inform the correct target date.

My tip for periodic routines is to leave the target date parameter with the default for the previous period. : Smirk:

13. Ensure ordering

In situations where order is important, it is always important to always use a single attribute as a tie-breaking criterion.

When we’re not careful about it, we’re usually only going to find the production error by using dummy data in the tests and rarely cover cases like this.

My tip for this item is to invest in prevention. : Innocent:

14. Beware of where by: updated_at and batches

The :updated_atattribute changes very frequently and between one SQL from one batch to another it can be updated and end up processing repeated records or failing to process others.

Unfortunately I have not found an optimal solution for this case but pre-selecting the IDs and then iterating in batches on the ids (which is fixed) ensures that all records are processed correctly only 1 times.

Trying to understand all the working and possible exceptions of these queries is very difficult then, in doubt, avoid batches with updated_at. : Sweat_smile:

Date and time

15. TimeZone

You thought you were not going to have anything to do with time?

The most common mistake is not to consider timezone in operations with date and time.

Even if your system does not need to handle timezones always use timezone so you do not waste time after fixing everything : Hourglass:

16. Very careful with Query Timezones

When making queries with ActiveRecord, it considers the timezone correctly so no problem.

But sometimes we need to make more “manual” queries, and in those cases that timezone care is all yours.

Use queries with parameters and o Time.zone to ensure: +1:

17. DB always works with UTC

The important thing here is that when you are going to generate a SQL in hand, you need to leave all dates in UTC. : Globe_with_meridians:

18. Spread the time

Often we receive a call in the API and we queue a Backgroud Job to complete the action.

When it is an action that creates something, it is very important that you record the date / time that the API was called.

This ensures that the object is saved with the correct date even if it has a delay in execution because of queues or slowness on the server. : Memo:

Importation exportation

Transporting data between systems is always a challenge.

We invented APIs, services, and other ways but still fall into the good old data transport using simple tables with comma-separated (.csv) data.

So when dealing with transport via csv, remember:

19. Do not let a case interrupt the whole process.

Has it ever happened to be running that migration script and locking it in the middle because it popped an exception or something and had to process it all over again?

Avoid this by ensuring that even if one entry gives some error the rest runs smoothly.

It’s better to have some unprocessed entries than to have everything else missing.

Also put mechanisms in place so that these exceptions are communicated to you and can be solved in some way.

Do not just add to the logs, something has to alert you, be it an email, an alert on your dashboard, anything.

If it is a very specific case, it may be easier for you to simply treat it in hand but if it is something that can affect more cases, hurry to correct it! : Runner:

20. Use “tab” as a separator in CSV

For one reason or another, one day we have to import or export data in CSV format.

In export, if we choose comma as separator and have some value inserted by the comma-containing user, we have to treat this so as not to break the CSV.

The most common is to surround the fields of the type open with quotation marks ( “) and if it has quotation marks in the value we have to treat it too and so on.

The code for all this is somewhat complex, increasing the chance of bugs, unforeseen cases and possibly having a degraded performance.

If we use the “tab” character as a separator the scenario changes.

Treating a “tab” inserted by the user by replacing it with “space” is practically imperceptible in most cases and we do not have to worry about any other character, the generated “CSV” is very clean and readable.

Of course there are some cases in which the user’s “tab” is important so we always have to think before choosing.

Trust me, the “tab” is your friend.

21. Treat the data

Who has never suffered with “equal” texts being considered different because of uppercase and lowercase, spaces at the beginning and end, etc.?

In the United States the date is in the month, day and year format, most of other countries it is day, month and year. When you simply read, you can easily end up changing the day and the month.

And that user who fills out a form with ” so- and-so + 2example.com ” and 2 different users for the same person?

There are many problems caused by this kind of nonsense, so before you go out filling the CSV of an export or read from an import treat them to ensure their integrity and be happy. : Sunglasses:

22. Validate the data

Even with everything being what it should be we sometimes receive dates from before Christ or in the future.

Null or invalid objects pop up several exceptions.

This is simple so you do not have to do it. :blush:

23. Encoding is Evil

I do not think I need to convert anyone whose encoding is evil.

This will help you in this battle.

Even so, it’s not a silver bullet, so if it’s confidence not too loud, it’s worth telling the user that the file could not be read and asked to convert to UTF-8 or some accepted format: Mag:

24. CSV with header: true

When reading a CSV with header, you must ignore the first line and ensure that the user sorts the columns of the file the way we want.

We know this does not happen, there will always be a CSV with the column in the wrong order.

Because of this we end up having to read the header and interpret each line accordingly.

There is a parameter in the CSV that already does this for you.

With the option headers: trueI can iterate over a CSV already with an object similar to a hash.

It’s free, enjoy and use : Raised_hands:

25. CSV Lint

The csvlint gem statically validates the CSV file.

It is so cool that it already returns all the errors indicating the reason and the error line.

Super easy and fast.

This eliminates much of the import errors. Gun

26. Show user errors

Okay, you’ve protected your system well by validating encoding, checking for file syntax errors, and bypassing rows with errors.

Your system may be 100% ok but the user has not yet had 100% of the important data imported.

Whatever bug you detect and ignore, let your dear user know in a very friendly way and giving tips on how he can fix it.

If possible, even generate a new CSV with only the lines with errors.

Help him and earn his loyalty. : Heart:

We still need to worry about files

27. Parameterize

Files are saved on various operating systems for users and servers.

Especially for us, accents and strange characters can cause a lot of headaches.

Fortunately parameterize is a good remedy for this and has no contraindication: Pill:

28. Avoid large names in files

Depending on the FileSystem or communication protocol, large file names can simply be cut off.

The FTP protocol, for example, does this.

I usually like to use meaningful names and sometimes end up getting big.

I already had to cut short and cut everything because I had problems with big names. : Scream:

29. Compact the CSV

CSV files tend to have many repeated characters, so compacting the size is reduced dramatically.

I saw files of 32Mb be reduced to 6Mb so it’s worth it : Package:

30. zipRuby, not RubyZip

RubyZip is 2x slower than zipRuby and allocates 700x more objects in memory

I do not think I need to say anything else.

Got any questions?

Do you have killer tips on the subject?

Leave a message in the comments!

Host WordPress on Amazon Cloud in a few clicks
Includes SSL, PHP7, Migration and More

No Credit Card Required for Sign Up.

x