truncation in capybara tests results in rspec freezing (while deletion works)

[steookk]

I found out that this code (necessary to truncate in case we are testing with capybara) makes RSpec freeze right after running capybara tests, while the same code with :deletion strategy works smoothly.

    config.before(:suite) do
      DatabaseCleaner.strategy = :transaction
    end

    config.around(:each) do |example|
      if example.metadata[:type] == :feature
        example.run
        DatabaseCleaner.clean_with :truncation #ONLY DELETION WORKS!
      else 
        DatabaseCleaner.start
        example.run
        DatabaseCleaner.clean
      end
    end

As a matter of fact, the following code works for truncation, which makes me think that DatabaseCleaner.start must be run for truncation as well.

    config.before(:each) do
      DatabaseCleaner.strategy = :transaction
    end
    config.before(:each, :type => :feature) do
      DatabaseCleaner.strategy = :truncation
    end
    config.before(:each) do
      DatabaseCleaner.start #runs for both strategies
    end

    config.after(:each) do
      DatabaseCleaner.clean
    end

I don't know whether this is a bug or a some kinds of choice due to some reasons, however I felt it was worth pointing it out.

[SimonKaluza]

We're having similar issues, but only when we use capybara-webkit as well... It seems to only affect 'js:true' testcases. Even on truncation our tests will (rarely but consistently) hang forever (inspecting the call stack/debugging shows that it hangs while starting the Capybara Brick server). This thread https://groups.google.com/forum/#!topic/ruby-capybara/sgBkBKVXf_w seems to indicate that it might have something to do with the way I'm intermingling Database checks with Javascript capybara interactions, but I've been unable to isolate them.

[scottwillson]

At least for me, this was a deadlock between DatabaseCleaner truncating tables and RSpec starting a transaction for the test. Not sure of the RSpec conventions here, but adding this to spec_helper.rb fixed it for me:

RSpec.configure do |config|
   config.use_transactional_fixtures = false
end

I'm using Postgres. Not sure that matters. In any case, the idea is to not mix DatabseCleaner truncation or deletion with RSpec transactions.

[thomasfedb]

I'm also seeing database_cleaner hang when DatabaseCleaner.clean is executed.

My database_cleaner.rb is:

RSpec.configure do |config|
  config.before(:suite) do
    DatabaseCleaner.strategy = :transaction
    DatabaseCleaner.clean_with :truncation
  end

  config.around(:each) do |example|
    if example.metadata[:js]
      DatabaseCleaner.strategy = :truncation
    else
      DatabaseCleaner.strategy = :transaction
    end

    DatabaseCleaner.start
    example.run
    DatabaseCleaner.clean
  end
end

[sevenseacat]

Also seeing a lot of hanging behaviour. The deletion config in the original post works, but a lot of other configs (including working configs copied from other projects) simply don't anymore.

edit: It doesn't seem to clean up after features properly though.... getting failing tests due to leftover data from previous features

[jfine]

Not sure if this is helpful but we (@vormwald) had a weird hanging issue as well.

Changing...

config.around(:each) do |example|
  DatabaseCleaner.strategy = example.metadata[:js] ? :deletion : :transaction
  DatabaseCleaner.cleaning do
    example.run
  end
end

...to...

config.before(:each) do |example|
  DatabaseCleaner.strategy = example.metadata[:js] ? :deletion : :transaction
  DatabaseCleaner.start
end
config.after(:each) do
  DatabaseCleaner.clean
end

...fixed our issue.

[aripollak]

I think this might be related to #292. I was running into this on Rails 4.1 when running a sequence of js, then non-js, then js specs again. Switching to always using truncation just resulted in more intermittent database inconsistency issues instead of just a consistent hang.

Disconnecting all connections at the end of the around block seems to have fixed this hanging problem, plus some other intermittent deadlock problems, though it might just be introducing enough of a delay for the threads to sort themselves out.

EDIT: Nevermind, this didn't actually fix it!

    config.around do |example|
      use_truncation_strategy = example.metadata[:js] || example.metadata[:multithread]
      DatabaseCleaner.strategy = use_truncation_strategy ? :deletion : :transaction

      DatabaseCleaner.cleaning do
        example.run
      end

      ActiveRecord::Base.connection_pool.disconnect!
    end

[sha1sum]

Switching from around to before and after as @jfine indicated has also solved my issue. I originally was using before and after but was experiencing deadock, so I used the connection pool disconnection from @aripollak as a (hopefully) preventative measure against that:

  config.before(:each) do |spec|
    DatabaseCleaner.strategy = spec.metadata[:js] ? :deletion : :transaction
    DatabaseCleaner.start
  end

  config.after(:each) do |spec|
    DatabaseCleaner.clean
    begin
      ActiveRecord::Base.connection.send :rollback_transaction_records, true
    rescue
    end
    ActiveRecord::Base.connection_pool.disconnect! if spec.metadata[:js]
  end

[lucashungaro]

Had this bug with Rails 4.1.8 + RSpec 3.1.0 + Turnip 1.2.4 + Capybara 2.4.4 + Poltergeist 1.5.1 + DatabaseCleaner 1.3.0 + Postgres 9.3 using an around filter to switch over cleaning strategies. Dozens of database transactions were left open, eventually making so no specs could execute anymore (the rspec runner just hang there waiting forever).

The solution was simply to using a before(:each) + after(:each) filter under RSpec.configure like this:

  config.before(:each) do |example|
    if [:feature, :request].include? example.metadata[:type]
      DatabaseCleaner.strategy = :truncation
    else
       DatabaseCleaner.strategy = :transaction
    end

    DatabaseCleaner.start
  end

  config.after(:each) do |example|
    DatabaseCleaner.clean
  end

[sevenseacat]

Unfortunately nothing here has fixed the problem in my app.

The test running sits there doing nothing, and when I start a Postgres console and check the pg_stat_activity table, I see the two connections sitting there, one active and one idle in transaction doing nothing. I don't know what's going on.

[lucashungaro]

@Karpah Can you paste your spec_helper here?

[airatdev]

I am experiencing the same issue:

This config causes specs to freeze:

RSpec.configure do |config|
  config.before(:suite) do
    DatabaseCleaner.strategy = :transaction
    DatabaseCleaner.clean_with(:truncation)
  end

  config.around(:each) do |example|
    DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction

    DatabaseCleaner.cleaning do
      example.run
    end
  end
end

Example provided by @lucashungaro fixes the issue, however I'm not sure why this happens. Any ideas?

[lucashungaro]

@airatshigapov It's probably an issue with around filters not working properly, so the transactions are never closed.

[ka8725]

May be this solution will help you: teamcapybara/capybara#1089 (comment)

[claptimes5]

I believe this is an issue with the transactions not being closed. I had an issue deletion and truncation and was able to hack a solution by rolling back open transactions before deleting or truncating my tables

claptimes5@347b74b

This should be taken care of in the Transaction clean method, but I haven't had time yet to determine a solution.

[DexterTheDragon]

I'm having the same experience that @lucashungaro mentions. Using an around hook ends up causing rspec to hang. Switching to a before and after hook works fine.

[tgaff]

Experiencing same behavior. My config looks very much like airatshigapov's and I can confirm that dropping the around block in favor of before and after seems to resolve the issue. The hang reported as a db deadlock in sqlite.
rails 4.2.1 + database_cleaner 1.4.1 + capybara 2.4.4 + rspec 3.2.3

[RyanVasichko]

I managed to solve the problem by taking out require 'capybara/rspec' and using this for my database cleaning:

config.use_transactional_fixtures = false

config.before(:suite) do
  DatabaseCleaner.clean_with(:truncation)
end

config.before(:each) do |example|
  DatabaseCleaner.strategy = example.metadata[:js] ? :deletion : :transaction
  DatabaseCleaner.start
end

config.after(:each) do
  Capybara.reset_sessions!
  DatabaseCleaner.clean
end

[alininja]

This thread helped me to find a fix for my problem, so I'll just post the fix here in case it's helpful for anyone:

config.around(:each) do |example|
    DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction
    DatabaseCleaner.cleaning do
      example.run
    end
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

I'm not sure why the extra DatabaseCleaner.clean was necessary, but the dead lock issue came back without it. I'm also not sure why it was necessary to run the spec inside the DatabaseCleaner.cleaning block; using just DatabaseCleaner.start and DatabaseCleaner.clean didn't work.

Just a bit more background on the issue I encounter since the above might not solve all the issues expressed on this thread:

Some feature specs were either taking too long or not exiting correctly, so it left a lock on the database, which caused all subsequent database-dependent specs to run extremely slow and then fail with this error:

Failure/Error: Unable to find matching line from backtrace
ActiveRecord::StatementInvalid:
  Mysql2::Error: Lock wait timeout exceeded; try restarting transaction: DELETE FROM `table_name`

[claptimes5]

Using

config.around(:each) do |example|
  DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction
  DatabaseCleaner.cleaning do
      example.run
  end
end

and the code in my pull request #364 I was able to resolve this issue where open transactions were left on various connections to the database. Can someone else see if this helps?

[karlingen]

@claptimes5 you deserve a medal. I've been pulling my hair of for the last 8 hours because of this issue. Your PR fixed it. Thanks a bunch!!

Added this to my Gemfile:

gem 'database_cleaner', github: 'DatabaseCleaner/database_cleaner', ref: 'b8edac6bd04fb89a267201fa8d47066d511fd9de'

[claptimes5]

@karlingen thanks!! I guess this issue can be closed? @etagwerker

[etagwerker]

@claptimes5 You're right sir. Thank you!

[rafaelsales]

@etagwerker @bmabey @JonRowe plans to have this fix rolled out? I'm currently using master in my gemfile :/

[karlingen]

@rafaelsales instead of using master you could lock it to a certain commit.
I have it like this while waiting for a new version:

gem 'database_cleaner', github: 'DatabaseCleaner/database_cleaner', ref: 'b8edac6bd04fb89a267201fa8d47066d511fd9de'

[etagwerker]

@rafaelsales Here it is: http://databasecleaner.github.io/2015/09/01/v1-5-0-is-out/ but @karlingen's suggestion should've worked before this release.