From b57a0845b17a6e3a7adf98e34b859d88bafdba51 Mon Sep 17 00:00:00 2001 From: Srinivas Raghunathan Date: Tue, 6 Feb 2018 13:40:39 -0800 Subject: [PATCH 1/7] add information on performance methodology --- README.md | 2 +- performance_methodology.md | 44 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 performance_methodology.md diff --git a/README.md b/README.md index d198dbdb..bbfb8d3d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ A lightning fast [JSON:API](http://jsonapi.org/) serializer for Ruby Objects. # Performance Comparison -We compare serialization times with Active Model Serializer as part of RSpec performance tests included on this library. We want to ensure that with every change on this library, serialization time is at least `25 times` faster than Active Model Serializers on up to current benchmark of 1000 records. +We compare serialization times with Active Model Serializer as part of RSpec performance tests included on this library. We want to ensure that with every change on this library, serialization time is at least `25 times` faster than Active Model Serializers on up to current benchmark of 1000 records. Please read the [performance document](https://github.com/Netflix/fast_jsonapi/blob/master/performance_methodology.md) for any questions related to methodology. ## Benchmark times for 250 records diff --git a/performance_methodology.md b/performance_methodology.md new file mode 100644 index 00000000..e1de9897 --- /dev/null +++ b/performance_methodology.md @@ -0,0 +1,44 @@ +# Performance using Fast JSON API + +We have been getting a few questions on Github about [Fast JSON API’s](https://github.com/Netflix/fast_jsonapi) performance statistics and the methodology used to measure the performance. This article is an attempt at addressing this aspect of the gem. + +## Prologue + +With use cases like infinite scroll on complex models and bulk update on index pages, we started observing performance degradation on our Rails APIs. Our first step was to enable instrumentation and then tune for performance. We realized that, on average, more than 50% of the time was being spent on AMS serialization. At the same time, we had a couple of APIs that were simply proxying requests on top of a non-Rails, non-JSON API endpoint. Guess what? The non-Rails endpoints were giving us serialized JSON back in a fraction of the time spent by AMS. + +This led us to explore AMS documentation in depth in an effort to try a variety of techniques such as caching, using OJ for JSON string generation etc. It didn’t yield the consistent results we were hoping to get. We loved the developer experience of using AMS, but wanted better performance for our use cases. + +We came up with patterns that we can rely upon such as: + +* We always use [JSON:API](http://jsonapi.org/) for our APIs +* We almost always serialize a homogenous list of objects (Example: An array of movies) + +On the other hand: + +* AMS is designed to serialize JSON in several different formats, not just JSON:API +* AMS can also handle lists that are not homogenous + +This led us to build our own object serialization library that would be faster because it would be tailored to our requirements. The usage of fast_jsonapi internally on production environments resulted in significant performance gains. + +## Benchmark Setup + +The benchmark setup is simple with classes for ``` Movie, Actor, MovieType, User ``` on ```movie_context.rb``` for fast_jsonapi serializers and on ```ams_context.rb``` for AMS serializers. We benchmark the serializers with ```1, 25, 250, 1000``` movies, then we output the result. We also ensure that JSON string output is equivalent to ensure neither library is doing excess work compared to the other. Please checkout [object_serializer_performance_spec](https://github.com/Netflix/fast_jsonapi/blob/master/spec/lib/object_serializer_performance_spec.rb). + +## Benchmark Results + +We benchmarked results for creating a Ruby Hash. This approach removes the effect of chosen JSON string generation engines like OJ, Yajl etc. Benchmarks indicate that fast_jsonapi consistently performs around ```25 times``` faster than AMS in generating a ruby hash. + +We applied a similar benchmark on the operation to serialize the objects to a JSON string. This approach helps with ensuring some important criterias, such as: + +* OJ is used as the JSON engine for benchmarking both AMS and fast_jsonapi +* The benchmark is easy to understand +* The benchmark helps to improve performance +* The benchmark influences design decisions for the gem + +This gem is currently used in several APIs at Netflix and has reduced the response times by more than half on many of these APIs. We truly appreciate the Ruby and Rails communities and wanted to contribute in an effort to help improve the performance of your APIs too. + +## Epilogue + +[Fast JSON API](https://github.com/Netflix/fast_jsonapi) is not a replacement for AMS. AMS is a great gem, and it does many things and is very flexible. We still use it for non JSON:API serialization and deserialization. What started off as an internal performance exercise evolved into fast_jsonapi and created an opportunity to give something back to the awesome **Ruby and Rails communities**. + +We are excited to share it with all of you since we believe that there will be **no** end to this need for speed on APIs. :) From 9bdd7322bd1db9e34dfecb69afc71b09226349cf Mon Sep 17 00:00:00 2001 From: Srinivas Raghunathan Date: Thu, 8 Feb 2018 14:38:43 -0800 Subject: [PATCH 2/7] add oss metadata --- OSSMETADATA | 1 + 1 file changed, 1 insertion(+) create mode 100644 OSSMETADATA diff --git a/OSSMETADATA b/OSSMETADATA new file mode 100644 index 00000000..b96d4a4d --- /dev/null +++ b/OSSMETADATA @@ -0,0 +1 @@ +osslifecycle=active From be491709ba9d06d4f5541aa8778d1b79ac4b5e51 Mon Sep 17 00:00:00 2001 From: Les Fletcher Date: Sat, 17 Feb 2018 11:55:59 -0800 Subject: [PATCH 3/7] rework of AS Notifications --- lib/fast_jsonapi/instrumentation.rb | 2 + .../instrumentation/serializable_hash.rb | 15 ++++ .../instrumentation/serialized_json.rb | 15 ++++ lib/fast_jsonapi/object_serializer.rb | 21 +----- .../as_notifications_negative_spec.rb | 56 +++++++++++++++ .../instrumentation/as_notifications_spec.rb | 69 +++++++++++++++++++ 6 files changed, 160 insertions(+), 18 deletions(-) create mode 100644 lib/fast_jsonapi/instrumentation.rb create mode 100644 lib/fast_jsonapi/instrumentation/serializable_hash.rb create mode 100644 lib/fast_jsonapi/instrumentation/serialized_json.rb create mode 100644 spec/lib/instrumentation/as_notifications_negative_spec.rb create mode 100644 spec/lib/instrumentation/as_notifications_spec.rb diff --git a/lib/fast_jsonapi/instrumentation.rb b/lib/fast_jsonapi/instrumentation.rb new file mode 100644 index 00000000..936aeb59 --- /dev/null +++ b/lib/fast_jsonapi/instrumentation.rb @@ -0,0 +1,2 @@ +require 'fast_jsonapi/instrumentation/serializable_hash' +require 'fast_jsonapi/instrumentation/serialized_json' diff --git a/lib/fast_jsonapi/instrumentation/serializable_hash.rb b/lib/fast_jsonapi/instrumentation/serializable_hash.rb new file mode 100644 index 00000000..eb86a60e --- /dev/null +++ b/lib/fast_jsonapi/instrumentation/serializable_hash.rb @@ -0,0 +1,15 @@ +require 'active_support/notifications' + +module FastJsonapi + module ObjectSerializer + + alias_method :serializable_hash_without_instrumentation, :serializable_hash + + def serializable_hash + ActiveSupport::Notifications.instrument(SERIALIZABLE_HASH_NOTIFICATION, { name: self.class.name }) do + serializable_hash_without_instrumentation + end + end + + end +end diff --git a/lib/fast_jsonapi/instrumentation/serialized_json.rb b/lib/fast_jsonapi/instrumentation/serialized_json.rb new file mode 100644 index 00000000..4fa3cb57 --- /dev/null +++ b/lib/fast_jsonapi/instrumentation/serialized_json.rb @@ -0,0 +1,15 @@ +require 'active_support/notifications' + +module FastJsonapi + module ObjectSerializer + + alias_method :serialized_json_without_instrumentation, :serialized_json + + def serialized_json + ActiveSupport::Notifications.instrument(SERIALIZED_JSON_NOTIFICATION, { name: self.class.name }) do + serialized_json_without_instrumentation + end + end + + end +end diff --git a/lib/fast_jsonapi/object_serializer.rb b/lib/fast_jsonapi/object_serializer.rb index 9ee80e95..971a468c 100644 --- a/lib/fast_jsonapi/object_serializer.rb +++ b/lib/fast_jsonapi/object_serializer.rb @@ -3,30 +3,15 @@ require 'active_support/inflector' require 'fast_jsonapi/serialization_core' -begin - require 'skylight' - SKYLIGHT_ENABLED = true -rescue LoadError - SKYLIGHT_ENABLED = false -end - module FastJsonapi module ObjectSerializer extend ActiveSupport::Concern include SerializationCore - included do - # Skylight integration - # To remove Skylight - # Remove the included do block - # Remove the Gemfile entry - if SKYLIGHT_ENABLED - include Skylight::Helpers - - instrument_method :serializable_hash - instrument_method :to_json - end + SERIALIZABLE_HASH_NOTIFICATION = 'render.fast_jsonapi.serializable_hash'.freeze + SERIALIZED_JSON_NOTIFICATION = 'render.fast_jsonapi.serialized_json'.freeze + included do # Set record_type based on the name of the serializer class set_type(reflected_record_type) if reflected_record_type end diff --git a/spec/lib/instrumentation/as_notifications_negative_spec.rb b/spec/lib/instrumentation/as_notifications_negative_spec.rb new file mode 100644 index 00000000..6912ed49 --- /dev/null +++ b/spec/lib/instrumentation/as_notifications_negative_spec.rb @@ -0,0 +1,56 @@ +require 'spec_helper' + +describe FastJsonapi::ObjectSerializer do + include_context 'movie class' + + context 'instrument' do + + before(:each) do + options = {} + options[:meta] = { total: 2 } + options[:include] = [:actors] + + @serializer = MovieSerializer.new([movie, movie], options) + end + + context 'serializable_hash' do + + it 'should send not notifications' do + events = [] + + ActiveSupport::Notifications.subscribe(FastJsonapi::ObjectSerializer::SERIALIZABLE_HASH_NOTIFICATION) do |*args| + events << ActiveSupport::Notifications::Event.new(*args) + end + + serialized_hash = @serializer.serializable_hash + + expect(events.length).to eq(0) + + expect(serialized_hash.key?(:data)).to eq(true) + expect(serialized_hash.key?(:meta)).to eq(true) + expect(serialized_hash.key?(:included)).to eq(true) + end + + end + + context 'serialized_json' do + + it 'should send not notifications' do + events = [] + + ActiveSupport::Notifications.subscribe(FastJsonapi::ObjectSerializer::SERIALIZED_JSON_NOTIFICATION) do |*args| + events << ActiveSupport::Notifications::Event.new(*args) + end + + json = @serializer.serialized_json + + expect(events.length).to eq(0) + + expect(json.length).to be > 50 + end + + end + + end + +end diff --git a/spec/lib/instrumentation/as_notifications_spec.rb b/spec/lib/instrumentation/as_notifications_spec.rb new file mode 100644 index 00000000..27bfdad8 --- /dev/null +++ b/spec/lib/instrumentation/as_notifications_spec.rb @@ -0,0 +1,69 @@ +require 'spec_helper' +require 'fast_jsonapi/instrumentation' + +describe FastJsonapi::ObjectSerializer do + include_context 'movie class' + + context 'instrument' do + + before(:each) do + options = {} + options[:meta] = { total: 2 } + options[:include] = [:actors] + + @serializer = MovieSerializer.new([movie, movie], options) + end + + context 'serializable_hash' do + + it 'should send notifications' do + events = [] + + ActiveSupport::Notifications.subscribe(FastJsonapi::ObjectSerializer::SERIALIZABLE_HASH_NOTIFICATION) do |*args| + events << ActiveSupport::Notifications::Event.new(*args) + end + + serialized_hash = @serializer.serializable_hash + + expect(events.length).to eq(1) + + event = events.first + + expect(event.duration).to be > 0 + expect(event.payload).to eq({ name: 'MovieSerializer' }) + expect(event.name).to eq(FastJsonapi::ObjectSerializer::SERIALIZABLE_HASH_NOTIFICATION) + + expect(serialized_hash.key?(:data)).to eq(true) + expect(serialized_hash.key?(:meta)).to eq(true) + expect(serialized_hash.key?(:included)).to eq(true) + end + + end + + context 'serialized_json' do + + it 'should send notifications' do + events = [] + + ActiveSupport::Notifications.subscribe(FastJsonapi::ObjectSerializer::SERIALIZED_JSON_NOTIFICATION) do |*args| + events << ActiveSupport::Notifications::Event.new(*args) + end + + json = @serializer.serialized_json + + expect(events.length).to eq(1) + + event = events.first + + expect(event.duration).to be > 0 + expect(event.payload).to eq({ name: 'MovieSerializer' }) + expect(event.name).to eq(FastJsonapi::ObjectSerializer::SERIALIZED_JSON_NOTIFICATION) + + expect(json.length).to be > 50 + end + + end + + end + +end From 2a4cce544a7469438b0e0d24b7e788c1a9cd0f95 Mon Sep 17 00:00:00 2001 From: Les Fletcher Date: Sat, 17 Feb 2018 14:31:20 -0800 Subject: [PATCH 4/7] taking a crack at the normalizers --- lib/fast_jsonapi/instrumentation/skylight.rb | 2 ++ .../skylight/normalizers/serializable_hash.rb | 22 +++++++++++++++++++ .../skylight/normalizers/serialized_json.rb | 22 +++++++++++++++++++ 3 files changed, 46 insertions(+) create mode 100644 lib/fast_jsonapi/instrumentation/skylight.rb create mode 100644 lib/fast_jsonapi/instrumentation/skylight/normalizers/serializable_hash.rb create mode 100644 lib/fast_jsonapi/instrumentation/skylight/normalizers/serialized_json.rb diff --git a/lib/fast_jsonapi/instrumentation/skylight.rb b/lib/fast_jsonapi/instrumentation/skylight.rb new file mode 100644 index 00000000..1f3dc7eb --- /dev/null +++ b/lib/fast_jsonapi/instrumentation/skylight.rb @@ -0,0 +1,2 @@ +require 'fast_jsonapi/instrumentation/skylight/normalizers/serializable_hash' +require 'fast_jsonapi/instrumentation/skylight/normalizers/serialized_json' diff --git a/lib/fast_jsonapi/instrumentation/skylight/normalizers/serializable_hash.rb b/lib/fast_jsonapi/instrumentation/skylight/normalizers/serializable_hash.rb new file mode 100644 index 00000000..bc27ee49 --- /dev/null +++ b/lib/fast_jsonapi/instrumentation/skylight/normalizers/serializable_hash.rb @@ -0,0 +1,22 @@ +require 'skylight' +require 'fast_jsonapi/instrumentation/serializable_hash' + +module FastJsonapi + module Instrumentation + module Skylight + module Normalizers + class SerializableHash < Skylight::Normalizers::Normalizer + + register FastJsonapi::ObjectSerializer::SERIALIZABLE_HASH_NOTIFICATION + + CAT = "view.#{FastJsonapi::ObjectSerializer::SERIALIZABLE_HASH_NOTIFICATION}".freeze + + def normalize(trace, name, payload) + [ CAT, payload[:name], nil ] + end + + end + end + end + end +end diff --git a/lib/fast_jsonapi/instrumentation/skylight/normalizers/serialized_json.rb b/lib/fast_jsonapi/instrumentation/skylight/normalizers/serialized_json.rb new file mode 100644 index 00000000..a04f6c0f --- /dev/null +++ b/lib/fast_jsonapi/instrumentation/skylight/normalizers/serialized_json.rb @@ -0,0 +1,22 @@ +require 'skylight' +require 'fast_jsonapi/instrumentation/serializable_hash' + +module FastJsonapi + module Instrumentation + module Skylight + module Normalizers + class SerializedJson < Skylight::Normalizers::Normalizer + + register FastJsonapi::ObjectSerializer::SERIALIZED_JSON_NOTIFICATION + + CAT = "view.#{FastJsonapi::ObjectSerializer::SERIALIZED_JSON_NOTIFICATION}".freeze + + def normalize(trace, name, payload) + [ CAT, payload[:name], nil ] + end + + end + end + end + end +end From e7c40af1eb020b441dd8b5ee547799e0aee7e38f Mon Sep 17 00:00:00 2001 From: Les Fletcher Date: Sat, 17 Feb 2018 14:31:33 -0800 Subject: [PATCH 5/7] Some docs for the instrumentation --- README.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/README.md b/README.md index bbfb8d3d..343e1dc2 100644 --- a/README.md +++ b/README.md @@ -244,6 +244,36 @@ object_method_name | Set custom method name to get related objects | ```has_many record_type | Set custom Object Type for a relationship | ```belongs_to :owner, record_type: :user``` serializer | Set custom Serializer for a relationship | ```has_many :actors, serializer: :custom_actor``` +### Instrumentation + +`fast_jsonapi` also has builtin [Skylight](https://www.skylight.io/) integration. To enable, add the following to an initializer: + +```ruby +require 'fast_jsonapi/instrumentation/skylight' +``` + +Skylight relies on `ActiveSupport::Notifications` to track these two core methods. If you would like to use these notifications without using Skylight, simply require the instrumentation integration: + +```ruby +require 'fast_jsonapi/instrumentation' +``` + +The two instrumented notifcations are supplied by these two constants: +* `FastJsonapi::ObjectSerializer::SERIALIZABLE_HASH_NOTIFICATION` +* `FastJsonapi::ObjectSerializer::SERIALIZED_JSON_NOTIFICATION` + +It is also possible to instrument one method without the other by using one of the following require statements: + +```ruby +require 'fast_jsonapi/instrumentation/serializable_hash' +require 'fast_jsonapi/instrumentation/serialized_json' +``` + +Same goes for the Skylight integration: +```ruby +require 'fast_jsonapi/instrumentation/skylight/normalizers/serializable_hash' +require 'fast_jsonapi/instrumentation/skylight/normalizers/serialized_json' +``` ## Contributing Please see [contribution check](https://github.com/Netflix/fast_jsonapi/blob/master/CONTRIBUTING.md) for more details on contributing From f2ec5e01879289c947baa58a93a5fbea73db3362 Mon Sep 17 00:00:00 2001 From: Les Fletcher Date: Tue, 27 Feb 2018 15:49:01 -0800 Subject: [PATCH 6/7] do two separate rspec runs because of require issue --- .travis.yml | 1 + .../{as_notifications_spec.rb => as_notifications.rb} | 0 2 files changed, 1 insertion(+) rename spec/lib/instrumentation/{as_notifications_spec.rb => as_notifications.rb} (100%) diff --git a/.travis.yml b/.travis.yml index 00b98f79..1a9864bf 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,3 +6,4 @@ rvm: - 2.5.0 script: - bundle exec rspec + - bundle exec rspec spec/lib/instrumentation/as_notifications.rb diff --git a/spec/lib/instrumentation/as_notifications_spec.rb b/spec/lib/instrumentation/as_notifications.rb similarity index 100% rename from spec/lib/instrumentation/as_notifications_spec.rb rename to spec/lib/instrumentation/as_notifications.rb From 1d529939ba14976490aa9ea401713caf5f2546b6 Mon Sep 17 00:00:00 2001 From: Les Fletcher Date: Thu, 8 Mar 2018 21:41:11 -0800 Subject: [PATCH 7/7] tear down the aliases --- .travis.yml | 1 - ..._notifications.rb => as_notifications_spec.rb} | 15 ++++++++++++++- 2 files changed, 14 insertions(+), 2 deletions(-) rename spec/lib/instrumentation/{as_notifications.rb => as_notifications_spec.rb} (80%) diff --git a/.travis.yml b/.travis.yml index 1a9864bf..00b98f79 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,4 +6,3 @@ rvm: - 2.5.0 script: - bundle exec rspec - - bundle exec rspec spec/lib/instrumentation/as_notifications.rb diff --git a/spec/lib/instrumentation/as_notifications.rb b/spec/lib/instrumentation/as_notifications_spec.rb similarity index 80% rename from spec/lib/instrumentation/as_notifications.rb rename to spec/lib/instrumentation/as_notifications_spec.rb index 27bfdad8..f7769ffe 100644 --- a/spec/lib/instrumentation/as_notifications.rb +++ b/spec/lib/instrumentation/as_notifications_spec.rb @@ -1,11 +1,24 @@ require 'spec_helper' -require 'fast_jsonapi/instrumentation' describe FastJsonapi::ObjectSerializer do include_context 'movie class' context 'instrument' do + before(:all) do + require 'fast_jsonapi/instrumentation' + end + + after(:all) do + [ :serialized_json, :serializable_hash ].each do |m| + alias_command = "alias_method :#{m}, :#{m}_without_instrumentation" + FastJsonapi::ObjectSerializer.class_eval(alias_command) + + remove_command = "remove_method :#{m}_without_instrumentation" + FastJsonapi::ObjectSerializer.class_eval(remove_command) + end + end + before(:each) do options = {} options[:meta] = { total: 2 }