checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
---
2
- SHA1:
3
- metadata.gz: 87d31c97612fa757fa73b7a87ab6d9372dfa1261
4
- data.tar.gz: feb0b94762924d848198740056a29c896fc2dbd5
2
+ SHA256:
3
+ metadata.gz: 57602c5018eb3d77a8a684949ffab67a73fed32f4a606e7c918f1cef8181e3b5
4
+ data.tar.gz: bae0f706bbbc0cdc301451630cb781d6f63b39ab23a54fad8ec12d3e9178de25
5
5
SHA512:
6
- metadata.gz: 495288c321b9aa8b2f6d77cd3b6c7028d3b1096a364346cc0473d81a8f4cb1e7a281210c6a8931187ec03a5b0ad03647999be5d101694ae548740af7f9742dce
7
- data.tar.gz: 7a7424667f0a053b83c18a22222df330dbbbb0ce8eca8cc3557493705d3b9860d309535e3ecf30de6fb89c1690bca5691b0eef9dbcc292c3ea54fd667e8b0940
6
+ metadata.gz: 134c2e5e7f8967167aef8e923c114b203b39e3ad0799de21d503346ac69b358495e52dec898934b322e2baef977c5bd16a2a989e68471cb0972c90ff3a4b07a4
7
+ data.tar.gz: 44b6700d349482e1d0fbd8d716299f821edd9fc9750bdffab746a949e0c80732e3747e8864f2594bdc79aa0abe40d73a9894c2712edacd12c33c454797be9235
data/.ruby-version CHANGED
@@ -1 +1 @@
1
- ruby-2.4.1
1
+ ruby-2.5.7
data/.travis.yml CHANGED
@@ -1,7 +1,7 @@
1
1
sudo: false
2
2
language: ruby
3
3
rvm:
4
- - 2.4.1
4
+ - 2.5.7
5
5
services:
6
6
- mysql
7
7
- postgresql
data/README.md CHANGED
@@ -8,28 +8,34 @@
8
8
9
9
Simple and robust sharding gem for Rails, including Migrations and ActiveRecord extensions
10
10
11
- This gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases. It also provides rake tasks and migrations to help you manage the schema by shard groups.
11
+ This gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases.
12
12
13
- After you have setup your shards, accessing them is as simple as:
13
+ Accessing shards is as simple as:
14
14
```ruby
15
+ # creating a user to a specific shard
15
16
new_user = User.using_shard(:shard_group1, :shard1).create(username: 'x')
17
+
18
+ # retrieving a user from a specific shard
16
19
loaded_user = User.using_shard(:shard_group1, :shard1).where(username: 'x').first
17
20
```
18
21
19
- You can also use the block syntax, where all your queries inside will be directed to the correct shard:
22
+ You can also use the block syntax:
20
23
```ruby
21
24
Rails::Sharding.using_shard(:shard_group1, :shard1) do
25
+ # All statements inside this block will go to the selected shard
26
+
27
+ # Do some queries
22
28
new_user = User.create(username: 'x')
23
29
loaded_user = User.where(username: 'x').first
24
30
billing_infos = loaded_user.billing_infos.all
25
31
end
26
32
```
27
33
28
- You can also pick and choose which models will be shardable, so that all the models that are not shardable will still be retrieved from the master database, even if inside a using_shard block.
34
+ You can also pick and choose which models will be shardable. Non shardable models will be retrieved from the master database, even if inside a `using_shard` block.
29
35
30
36
## Compatibility
31
37
Gem version 1.x.x:
32
- * Rails 5.0
38
+ * Rails 5.0 and 5.1
33
39
* Databases: MySQL, MariaDB, Postgres
34
40
35
41
Gem version 0.x.x:
@@ -51,7 +57,6 @@ bundle
51
57
```
52
58
53
59
## Creating Shards
54
- This gem helps you create shards that are additional and completely separate from your master database. The master database is the one that is created and managed through rails, and is the default storage for all your models.
55
60
56
61
To start with the rails-sharding gem, run the command
57
62
```
@@ -80,7 +85,9 @@ development:
80
85
...
81
86
```
82
87
83
- Rename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`). A shard group is simply a set of shards that should have the same schema.
88
+ Rename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`).
89
+
90
+ **A shard group is a set of shards that should have the same schema.**
84
91
85
92
When you're ready to create the shards run
86
93
```
@@ -88,7 +95,13 @@ rake shards:create
88
95
```
89
96
90
97
## Migrating Shards
91
- Go to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema. For example, add the following migration to your `db/shards_migrations/shard_group1`:
98
+ Go to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema.
99
+
100
+
101
+ As of now, there is no generator for migrations. You can use the regular rails generator and move the migrations to the `shards_migration` folder.
102
+
103
+
104
+ For example, add the following migration to your `db/shards_migrations/shard_group1`:
92
105
```ruby
93
106
# 20160808000000_create_users.rb
94
107
class CreateClients < ActiveRecord::Migration[5.0]
@@ -175,6 +188,9 @@ Rails::Sharding.setup do |config|
175
188
end
176
189
```
177
190
191
+ ## Wiki
192
+ Want to know more? How to integrate with RSpec, Capistrano, etc? Take a look at our [wiki](https://github.com/hsgubert/rails-sharding/wiki).
193
+
178
194
## Development and Contributing
179
195
180
196
After checking out the repo:
data/lib/rails/sharding/connection_handler.rb CHANGED
@@ -26,13 +26,30 @@ module Rails::Sharding
26
26
27
27
resolver = ActiveRecord::ConnectionAdapters::ConnectionSpecification::Resolver.new(shard_group_configurations)
28
28
begin
29
+ connection_spec = resolver.spec(shard_name.to_sym)
30
+
31
+ # since Rails 5.1 connection_spec already comes with :name set to whatever
32
+ # key you used to retrieve it from the resolver, in this case the shard_name.
33
+ # We don't want that, so we overwrite it with our connection name formed
34
+ # by shard_group:shard_name
29
35
connection_name = connection_name(shard_group, shard_name)
30
- connection_spec = resolver.spec(shard_name.to_sym, connection_name)
36
+ connection_spec.instance_variable_set(:@name, connection_name)
31
37
rescue ActiveRecord::AdapterNotSpecified
32
- raise Errors::ConfigNotFoundError, "Cannot find configuration for shard '#{shard_group}:#{shard_name}' in environment '#{environment}' in #{Config.shards_config_file}"
38
+ raise Errors::ConfigNotFoundError, "Cannot find configuration for shard '#{shard_group}:#{shard_name}' in environment '#{environment}' in #{Config.shards_config_file}, or it does not specify :adapter"
33
39
end
34
40
35
- connection_handler.establish_connection(connection_spec)
41
+ # Since Rails 5.1 we cannot use connection_handler.establish_connection anymore,
42
+ # because it does more than establishing_connection. It does a second
43
+ # connection specification lookup on Base.configurations (where our spec
44
+ # is not, because we define it in a shards.yml instead of the regular
45
+ # database.yml) and it notifies subscribers of a event that does not concern
46
+ # us. Because of this we directly create the connection_pool and inject to
47
+ # the handler.
48
+ # Note: we should consider writting our connection handler from scratch, since
49
+ # it has become simpler than reusing the one from rails. And it will not break
50
+ # as long as the ConnectionPool interface is stable.
51
+ connection_handler.remove_connection(connection_spec.name)
52
+ connection_handler.send(:owner_to_pool)[connection_spec.name] = ActiveRecord::ConnectionAdapters::ConnectionPool.new(connection_spec)
36
53
end
37
54
38
55
def self.connection_pool(shard_group, shard_name)
@@ -61,7 +78,7 @@ module Rails::Sharding
61
78
end
62
79
63
80
def self.with_connection(shard_group, shard_name, &block)
64
- connection_pool(shard_group, shard_name).with_connection do |connection|
81
+ connection_pool(shard_group, shard_name).with_connection do |connection|
65
82
if connection && Config.add_shard_tag_to_query_logs
66
83
connection_name = connection_name(shard_group, shard_name)
67
84
add_shard_tag_to_connection_log(connection, connection_name)
@@ -91,6 +108,8 @@ module Rails::Sharding
91
108
end
92
109
93
110
# Adds a shard tag to the log of all queries executed through this connection
111
+ # Obs: connection inherits from ActiveRecord::ConnectionAdapters::AbstractAdapter
112
+ # but its class depends on the database adapter used.
94
113
def self.add_shard_tag_to_connection_log(connection, shard_tag)
95
114
# avoids modifing connection twice
96
115
if connection.respond_to? :shard_tag
@@ -107,9 +126,9 @@ module Rails::Sharding
107
126
108
127
# defines a new #log that adds a tag to the log
109
128
class << connection
110
- def log(sql, name="SQL", binds=[], statement_name=nil, &block)
129
+ def log(sql, name="SQL", binds=[], type_casted_binds=[], statement_name=nil, &block)
111
130
name = (name.to_s + " (#{shard_tag})").strip
112
- self.original_log(sql, name, binds, statement_name, &block)
131
+ original_log(sql, name, binds, type_casted_binds, statement_name, &block)
113
132
end
114
133
end
115
134
data/lib/rails/sharding/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
module Rails
2
2
module Sharding
3
- VERSION = "1.0.3"
3
+ VERSION = "1.1.2"
4
4
end
5
5
end
data/lib/tasks/rails-sharding.rake CHANGED
@@ -274,22 +274,31 @@ shards_namespace = namespace :shards do
274
274
275
275
desc "Empty the test shards (drops all tables) (options: SHARD_GROUP=x, SHARD=x)"
276
276
task :purge => [:_make_activerecord_base_shardable] do
277
- Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
278
- puts "== Purging test shard #{shard_group}:#{shard}"
279
- begin
280
- # establishes connection with test shard, saving if it was connected before (rails 4.2 doesn't do this, but should)
277
+ begin
278
+ # saves the current RAILS_ENV (we must change it so the environment is set correcly on the metadata table)
279
+ initial_rails_env = Rails.env
280
+ Rails.env = 'test'
281
- should_reconnect = Rails::Sharding::ConnectionHandler.connection_pool(shard_group, shard).active_connection?
282
- Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard, 'test')
283
281
284
- Rails::Sharding.using_shard(shard_group, shard) do
285
- ActiveRecord::Tasks::DatabaseTasks.purge(configuration)
286
- end
287
- ensure
288
- if should_reconnect
289
- # reestablishes connection for RAILS_ENV environment (whatever that is)
290
- Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard)
282
+ Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
283
+ puts "== Purging test shard #{shard_group}:#{shard}"
284
+ begin
285
+ # establishes connection with test shard, saving if it was connected before (rails 4.2 doesn't do this, but should)
286
+ should_reconnect = Rails::Sharding::ConnectionHandler.connection_pool(shard_group, shard).active_connection?
287
+ Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard, 'test')
288
+
289
+ Rails::Sharding.using_shard(shard_group, shard) do
290
+ ActiveRecord::Tasks::DatabaseTasks.purge(configuration)
291
+ end
292
+ ensure
293
+ if should_reconnect
294
+ # reestablishes connection for RAILS_ENV environment (whatever that is)
295
+ Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard)
296
+ end
291
297
end
292
298
end
299
+ ensure
300
+ # restores rails env
301
+ Rails.env = initial_rails_env
293
302
end
294
303
end
295
304
end
data/rails-sharding.gemspec CHANGED
@@ -23,14 +23,14 @@ Gem::Specification.new do |spec|
23
23
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
24
24
spec.require_paths = ["lib"]
25
25
26
- spec.add_runtime_dependency 'rails', '5.0.2'
26
+ spec.add_runtime_dependency 'rails', '~> 5.1.0'
27
27
28
- spec.add_development_dependency "bundler", "~> 1.15"
29
- spec.add_development_dependency "rake", "~> 12.0"
28
+ spec.add_development_dependency "bundler", "~> 1.17"
29
+ spec.add_development_dependency "rake", "~> 13.0"
30
30
spec.add_development_dependency "rspec", "~> 3.0"
31
- spec.add_development_dependency "byebug", '~> 9'
31
+ spec.add_development_dependency "byebug", '~> 11'
32
32
spec.add_development_dependency "mysql2", '~> 0'
33
- spec.add_development_dependency "pg" # postgres driver
33
+ spec.add_development_dependency "pg", '~> 0' # postgres driver
34
34
spec.add_development_dependency "codeclimate-test-reporter", '~> 1'
35
- spec.add_development_dependency "simplecov"
35
+ spec.add_development_dependency "simplecov", '~> 0'
36
36
end
metadata CHANGED
@@ -1,57 +1,57 @@
1
1
--- !ruby/object:Gem::Specification
2
2
name: rails-sharding
3
3
version: !ruby/object:Gem::Version
4
- version: 1.0.3
4
+ version: 1.1.2
5
5
platform: ruby
6
6
authors:
7
7
- Henrique Gubert
8
8
autorequire:
9
9
bindir: exe
10
10
cert_chain: []
11
- date: 2017-09-13 00:00:00.000000000 Z
11
+ date: 2019-11-08 00:00:00.000000000 Z
12
12
dependencies:
13
13
- !ruby/object:Gem::Dependency
14
14
name: rails
15
15
requirement: !ruby/object:Gem::Requirement
16
16
requirements:
17
- - - '='
17
+ - - "~>"
18
18
- !ruby/object:Gem::Version
19
- version: 5.0.2
19
+ version: 5.1.0
20
20
type: :runtime
21
21
prerelease: false
22
22
version_requirements: !ruby/object:Gem::Requirement
23
23
requirements:
24
- - - '='
24
+ - - "~>"
25
25
- !ruby/object:Gem::Version
26
- version: 5.0.2
26
+ version: 5.1.0
27
27
- !ruby/object:Gem::Dependency
28
28
name: bundler
29
29
requirement: !ruby/object:Gem::Requirement
30
30
requirements:
31
31
- - "~>"
32
32
- !ruby/object:Gem::Version
33
- version: '1.15'
33
+ version: '1.17'
34
34
type: :development
35
35
prerelease: false
36
36
version_requirements: !ruby/object:Gem::Requirement
37
37
requirements:
38
38
- - "~>"
39
39
- !ruby/object:Gem::Version
40
- version: '1.15'
40
+ version: '1.17'
41
41
- !ruby/object:Gem::Dependency
42
42
name: rake
43
43
requirement: !ruby/object:Gem::Requirement
44
44
requirements:
45
45
- - "~>"
46
46
- !ruby/object:Gem::Version
47
- version: '12.0'
47
+ version: '13.0'
48
48
type: :development
49
49
prerelease: false
50
50
version_requirements: !ruby/object:Gem::Requirement
51
51
requirements:
52
52
- - "~>"
53
53
- !ruby/object:Gem::Version
54
- version: '12.0'
54
+ version: '13.0'
55
55
- !ruby/object:Gem::Dependency
56
56
name: rspec
57
57
requirement: !ruby/object:Gem::Requirement
@@ -72,14 +72,14 @@ dependencies:
72
72
requirements:
73
73
- - "~>"
74
74
- !ruby/object:Gem::Version
75
- version: '9'
75
+ version: '11'
76
76
type: :development
77
77
prerelease: false
78
78
version_requirements: !ruby/object:Gem::Requirement
79
79
requirements:
80
80
- - "~>"
81
81
- !ruby/object:Gem::Version
82
- version: '9'
82
+ version: '11'
83
83
- !ruby/object:Gem::Dependency
84
84
name: mysql2
85
85
requirement: !ruby/object:Gem::Requirement
@@ -98,14 +98,14 @@ dependencies:
98
98
name: pg
99
99
requirement: !ruby/object:Gem::Requirement
100
100
requirements:
101
- - - ">="
101
+ - - "~>"
102
102
- !ruby/object:Gem::Version
103
103
version: '0'
104
104
type: :development
105
105
prerelease: false
106
106
version_requirements: !ruby/object:Gem::Requirement
107
107
requirements:
108
- - - ">="
108
+ - - "~>"
109
109
- !ruby/object:Gem::Version
110
110
version: '0'
111
111
- !ruby/object:Gem::Dependency
@@ -126,14 +126,14 @@ dependencies:
126
126
name: simplecov
127
127
requirement: !ruby/object:Gem::Requirement
128
128
requirements:
129
- - - ">="
129
+ - - "~>"
130
130
- !ruby/object:Gem::Version
131
131
version: '0'
132
132
type: :development
133
133
prerelease: false
134
134
version_requirements: !ruby/object:Gem::Requirement
135
135
requirements:
136
- - - ">="
136
+ - - "~>"
137
137
- !ruby/object:Gem::Version
138
138
version: '0'
139
139
description: |-
@@ -193,8 +193,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
193
193
- !ruby/object:Gem::Version
194
194
version: '0'
195
195
requirements: []
196
- rubyforge_project:
196
+ rubygems_version: 3.0.6
197
- rubygems_version: 2.6.11
198
197
signing_key:
199
198
specification_version: 4
200
199
summary: Simple and robust sharding for Rails, including Migrations and ActiveRecord