Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add redis multi instance monitor support for redis-sentinel #356

Closed
wants to merge 49 commits into from
Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e85b6e8
Merge pull request #1 from voxpupuli/master
basti-nis May 5, 2020
facd408
add multi instance support for redis-sentinel
basti-nis May 5, 2020
c87be32
add multi instance support to sentinel
basti-nis May 5, 2020
a4d1b5b
set auth_pass to optional / spec changed
basti-nis May 5, 2020
dc95a13
fix sentinel spec
basti-nis May 5, 2020
c4230cf
TGTIT: Thx god there is Travis
basti-nis May 5, 2020
4744ae6
delete doubled auth_pass in type
basti-nis May 5, 2020
deb4075
close the given hash
basti-nis May 5, 2020
68f44c4
accidently added folder with wrong name
basti-nis May 5, 2020
1a4e9b6
fix closing class in acceptance test
basti-nis May 5, 2020
7e811fc
add parameter to fix full cover test
basti-nis May 5, 2020
dafbd47
fix redis_sentinel_spec
basti-nis May 5, 2020
c213e88
fix travis
basti-nis May 5, 2020
fef0597
new variable name / puppet strings
basti-nis May 6, 2020
c09d1ff
catch basic inclusion, dunno why it was run away
basti-nis May 6, 2020
d5482b4
confused about puppet strings
basti-nis May 6, 2020
e6d9cef
finishing sentinel instance monitor
basti-nis May 6, 2020
7846856
fix intending
basti-nis May 6, 2020
7f53eb9
Delete redis-sentinel.conf_master.epp
basti-nis May 6, 2020
2bc9ccf
fix acceptance check
basti-nis May 6, 2020
041ba16
defaults, merge fix
basti-nis May 7, 2020
b368f80
fix trailing comma
basti-nis May 7, 2020
38d19eb
some improvements suggested from ekohl
basti-nis Jun 4, 2020
b2e1a9e
fix trailing comma
basti-nis Jun 4, 2020
9d60184
fix defaults
basti-nis Jun 4, 2020
b36cc11
remove concat
basti-nis Nov 9, 2020
444933e
adding diffs from master
basti-nis Nov 9, 2020
e31e457
fixing spec
basti-nis Nov 9, 2020
4844efb
Merge branch 'master' into master
basti-nis Nov 9, 2020
bd0ae2e
remove remaining concat code
basti-nis Nov 9, 2020
ad146c9
readd comments
basti-nis Nov 9, 2020
0bf57f0
double param comment removed
basti-nis Nov 9, 2020
dc15fb9
remove dead parameter
basti-nis Nov 9, 2020
a688036
revert from ensure => present to file
basti-nis Nov 9, 2020
184ccdd
remove empty row
basti-nis Nov 9, 2020
88434c1
revert content to $conf_template
basti-nis Nov 9, 2020
170269a
there should be a single space or newline before a closing brace
basti-nis Nov 9, 2020
7298967
fix unit tests
basti-nis Nov 10, 2020
1e1e552
delete comma at the end of hash
basti-nis Nov 10, 2020
a8a20b7
protected mode failure workaround
basti-nis Nov 10, 2020
484ce19
fix syntax error in spec
basti-nis Nov 10, 2020
0fa0889
fix template and spec test
basti-nis Nov 10, 2020
3894fa4
fix duplicate entrys
basti-nis Nov 10, 2020
f417d65
add protected mode support
basti-nis Nov 10, 2020
204b844
fix spec test and protected mode
basti-nis Nov 10, 2020
e809d0a
minimum_version should be private
basti-nis Nov 10, 2020
38394bb
Merge pull request #2 from basti-nis/hotfix/protected_mode_fix
basti-nis Nov 10, 2020
d2d9bcc
remove comments
basti-nis Nov 10, 2020
eed6b76
add missing traling comma after last element
basti-nis Nov 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions manifests/params.pp
Original file line number Diff line number Diff line change
Expand Up @@ -200,4 +200,18 @@
fail "Operating system ${facts['os']['name']} is not supported yet."
}
}

$sentinel_master_name = {
'mymaster' => {
basti-nis marked this conversation as resolved.
Show resolved Hide resolved
redis_host => '127.0.0.1',
redis_port => 6379,
quorum => 2,
down_after => 30000,
parallel_sync => 1,
failover_timeout => 180000,
auth_pass => undef,
notification_script => undef,
client_reconfig_script => undef,
},
}
}
152 changes: 100 additions & 52 deletions manifests/sentinel.pp
Original file line number Diff line number Diff line change
@@ -1,7 +1,42 @@
# @summary Install redis-sentinel
#
# @param auth_pass
# The password to use to authenticate with the master and slaves.
# @param master_name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to define a type alias using a struct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean something like this:
type Redis::MasterName = Struct[{

instead of this:
type Redis::MasterName = Hash[String, Struct[{

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do u think about this:

type Redis::MasterName = Struct[{
  name                   => String,
  redis_host             => Stdlib::Host,
  redis_port             => Stdlib::Port,
  quorum                 => Integer[1],
  down_after             => Integer[1],
  parallel_sync          => Integer[0],
  failover_timeout       => Integer[1],
  auth_pass              => Optional[String],
  notification_script    => Optional[Stdlib::Absolutepath],
  client_reconfig_script => Optional[Stdlib::Absolutepath],
}]

The puppet code will look like:

     master_name   => {
       'session_6381' => {
         name             => 'redis-session-instance-6381',
         redis_host       => $redis_master_ip,
         redis_port       => 6381,
         quorum           => 2,
         parallel_sync    => 1,
         down_after       => 5000,
         failover_timeout => 12000,
         auth_pass        => $redis_auth,
       },
       'cache_6380' => {
         name             => 'redis-cache-instance-6380',
         redis_host       => $redis_master_ip,
         redis_port       => 6381,
         quorum           => 2,
         parallel_sync    => 1,
         down_after       => 5000,
         failover_timeout => 12000,
         auth_pass        => $redis_auth,
       },

The template would be changed to:

<%- | 
      $name,
      $redis_host,
      $redis_port,
      $quorum,
      $down_after,
      $parallel_sync,
      $failover_timeout,
      $auth_pass,
      $notification_script,
      $client_reconfig_script,
| -%>

sentinel monitor <%= $name%> <%= $redis_host %> <%= $redis_port %> <%= $quorum %>
sentinel down-after-milliseconds <%= $name%> <%= $down_after %>
sentinel parallel-syncs <%= $name%> <%= $parallel_sync %>
sentinel failover-timeout <%= $name%> <%= $failover_timeout %>
<% if $auth_pass { -%>
sentinel auth-pass <%= $name%> <%= $auth_pass %>
<% } -%>
<% if $notification_script { -%>
sentinel notification-script <%= $name%> <%= $notification_script %>
<% } -%>
<% if $client_reconfig_script { -%>
sentinel client-reconfig-script <%= $name%> <%= $client_reconfig_script %>
<% } -%>

Do you mean something like that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't realize you already had a struct. I was mostly thinking about the documentation, since you specify all keys. Would it be better to document the type instead?

My suggestion would be to generate the reference (rake strings:generate:reference) and see how they both look like. See what makes the most sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the variable name to make clear, that we have an breaking change with this feature.
The handling is a little bit more user friendly.

And i run puppet strings to document the data type.

# Specify the name of the master redis server.
# The valid charset is A-z 0-9 and the three characters ".-_".
# Hash with following Keys:
# @key auth_pass
# The password to use to authenticate with the master and slaves.
#
# @key redis_host
# Specify the bound host of the master redis server.
#
# @key redis_port
# Specify the port of the master redis server.
#
# @key daemonize
# Have Redis sentinel run as a daemon.
#
# @key down_after
# Number of milliseconds the master (or any attached slave or sentinel)
# should be unreachable (as in, not acceptable reply to PING, continuously,
# for the specified period) in order to consider it in S_DOWN state.
#
# @key failover_timeout
# Specify the failover timeout in milliseconds.
#
# @key parallel_sync
# How many slaves can be reconfigured at the same time to use a
# new master after a failover.
#
# @key quorum
# Number of sentinels that must agree that a master is down to
# signal sdown state.
#
# @key notification_script
# Path to the notification script
#
# @key client_reconfig_script
# Path to the client-reconfig script
#
# @param config_file
# The location and name of the sentinel config file.
Expand All @@ -19,17 +54,6 @@
# @param conf_template
# Define which template to use.
#
# @param daemonize
# Have Redis sentinel run as a daemon.
#
# @param down_after
# Number of milliseconds the master (or any attached slave or sentinel)
# should be unreachable (as in, not acceptable reply to PING, continuously,
# for the specified period) in order to consider it in S_DOWN state.
#
# @param failover_timeout
# Specify the failover timeout in milliseconds.
#
# @param init_script
# Specifiy the init script that will be created for sentinel.
#
Expand All @@ -43,29 +67,15 @@
# Specify the name of the master redis server.
# The valid charset is A-z 0-9 and the three characters ".-_".
#
# @param redis_host
# Specify the bound host of the master redis server.
#
# @param redis_port
# Specify the port of the master redis server.
#
# @param package_name
# The name of the package that installs sentinel.
#
# @param package_ensure
# Do we ensure this package.
#
# @param parallel_sync
# How many slaves can be reconfigured at the same time to use a
# new master after a failover.
#
# @param pid_file
# If sentinel is daemonized it will write its pid at this location.
#
# @param quorum
# Number of sentinels that must agree that a master is down to
# signal sdown state.
#
# @param sentinel_bind
# Allow optional sentinel server ip binding. Can help overcome
# issues arising from protect-mode added Redis 3.2
Expand All @@ -89,42 +99,49 @@
# The directory into which sentinel will change to avoid mount
# conflicts.
#
# @param notification_script
# Path to the notification script
#
# @param client_reconfig_script
# Path to the client-reconfig script
#
# @example Basic inclusion
# include redis::sentinel
#
# @example Configuring options
# class {'redis::sentinel':
# down_after => 80000,
# log_file => '/var/log/redis/sentinel.log',
# log_file => '/var/log/redis/sentinel.log',
# master_name => {
# 'session_6381' => {
# redis_host => $redis_master_ip,
# redis_port => 6381,
# quorum => 2,
# parallel_sync => 1,
# down_after => 5000,
# failover_timeout => 12000,
# auth_pass => $redis_auth,
# },
# 'cache_6380' => {
# redis_host => $redis_master_ip,
# redis_port => 6380,
# quorum => 2,
# parallel_sync => 1,
# down_after => 5000,
# failover_timeout => 12000,
# auth_pass => $redis_auth,
# }
# }
# }
#
class redis::sentinel (
Optional[String[1]] $auth_pass = undef,
Stdlib::Absolutepath $config_file = $redis::params::sentinel_config_file,
Stdlib::Absolutepath $config_file_orig = $redis::params::sentinel_config_file_orig,
Stdlib::Filemode $config_file_mode = '0644',
String[1] $conf_template = 'redis/redis-sentinel.conf.erb',
Boolean $daemonize = $redis::params::sentinel_daemonize,
Integer[1] $down_after = 30000,
Integer[1] $failover_timeout = 180000,
Optional[Stdlib::Absolutepath] $init_script = $redis::params::sentinel_init_script,
String[1] $init_template = 'redis/redis-sentinel.init.erb',
Redis::LogLevel $log_level = 'notice',
Stdlib::Absolutepath $log_file = $redis::params::sentinel_log_file,
String[1] $master_name = 'mymaster',
Stdlib::Host $redis_host = '127.0.0.1',
Stdlib::Port $redis_port = 6379,
Redis::MasterName $master_name = $redis::params::sentinel_master_name,
String[1] $package_name = $redis::params::sentinel_package_name,
String[1] $package_ensure = 'present',
Integer[0] $parallel_sync = 1,
Stdlib::Absolutepath $pid_file = $redis::params::sentinel_pid_file,
Integer[1] $quorum = 2,
Variant[Undef, Stdlib::IP::Address, Array[Stdlib::IP::Address]] $sentinel_bind = undef,
Stdlib::Port $sentinel_port = 26379,
String[1] $service_group = 'redis',
Expand All @@ -133,34 +150,65 @@
Boolean $service_enable = true,
String[1] $service_user = 'redis',
Stdlib::Absolutepath $working_dir = $redis::params::sentinel_working_dir,
Optional[Stdlib::Absolutepath] $notification_script = undef,
Optional[Stdlib::Absolutepath] $client_reconfig_script = undef,
) inherits redis::params {

require 'redis'

if $facts['os']['family'] == 'Debian' {
package { $package_name:
ensure => $package_ensure,
before => File[$config_file_orig],
before => Concat[$config_file_orig],
}

if $init_script {
Package[$package_name] -> File[$init_script]
}
}

file { $config_file_orig:
ensure => file,
owner => $service_user,
group => $service_group,
mode => $config_file_mode,
content => template($conf_template),
Concat {
owner => $service_user,
group => $service_group,
mode => $config_file_mode,
}

concat { $config_file_orig:
ensure => present,
}
basti-nis marked this conversation as resolved.
Show resolved Hide resolved

concat::fragment { 'sentinel_conf_header':
target => $config_file_orig,
order => 10,
content => template('redis/sentinel/redis-sentinel.conf_header.erb'),
}

$master_name.each |String $master, Hash $params| {
concat::fragment { "sentinel_conf_master_${master}" :
target => $config_file_orig,
order => 20,
content => epp('redis/sentinel/redis-sentinel.conf_master.epp', {
master_name => $master,
redis_host => $params['redis_host'],
redis_port => $params['redis_port'],
quorum => $params['quorum'],
down_after => $params['down_after'],
parallel_sync => $params['parallel_sync'],
failover_timeout => $params['failover_timeout'],
auth_pass => $params['auth_pass'],
notification_script => $params['notification_script'],
client_reconfig_script => $params['client_reconfig_script'],
}),
}
}

concat::fragment { 'sentinel_conf_footer':
target => $config_file_orig,
order => 30,
content => template('redis/sentinel/redis-sentinel.conf_footer.erb'),
}

exec { "cp -p ${config_file_orig} ${config_file}":
path => '/usr/bin:/bin',
subscribe => File[$config_file_orig],
subscribe => Concat[$config_file_orig],
notify => Service[$service_name],
refreshonly => true,
}
Expand Down
16 changes: 13 additions & 3 deletions spec/acceptance/suites/default/redis_sentinel_one_node_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,19 @@
it 'runs successfully' do
pp = <<-EOS
class { 'redis::sentinel':
master_name => 'mymaster',
redis_host => '127.0.0.1',
failover_timeout => 10000,
master_name => {
'mymaster' => {
redis_host => '127.0.0.1',
redis_port => 6379,
quorum => 2,
parallel_sync => 1,
down_after => 30000,
failover_timeout => 180000,
auth_pass => 'mymastertest',
notification_script => '/path/to/bar.sh',
client_reconfig_script => '/path/to/foo.sh',
},
},
}
EOS

Expand Down
21 changes: 14 additions & 7 deletions spec/classes/redis_sentinel_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -73,15 +73,22 @@
describe 'with custom parameters' do
let(:params) do
{
auth_pass: 'password',
master_name: {
'cow' => {
'redis_host' => '127.0.0.1',
'redis_port' => 6379,
'quorum' => 2,
'parallel_sync' => 1,
'auth_pass' => 'password',
'down_after' => 6000,
'failover_timeout' => 28_000,
'notification_script' => '/path/to/bar.sh',
'client_reconfig_script' => '/path/to/foo.sh'
}
},
sentinel_bind: '192.0.2.10',
master_name: 'cow',
down_after: 6000,
working_dir: '/tmp/redis',
log_file: '/tmp/barn-sentinel.log',
failover_timeout: 28_000,
notification_script: '/path/to/bar.sh',
client_reconfig_script: '/path/to/foo.sh'
log_file: '/tmp/barn-sentinel.log'
}
end

Expand Down
24 changes: 0 additions & 24 deletions templates/redis-sentinel.conf.erb

This file was deleted.

3 changes: 3 additions & 0 deletions templates/sentinel/redis-sentinel.conf_footer.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@

loglevel <%= @log_level %>
logfile <%= @log_file %>
7 changes: 7 additions & 0 deletions templates/sentinel/redis-sentinel.conf_header.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<% if @sentinel_bind -%>
bind <%= @sentinel_bind %>
<% end -%>
port <%= @sentinel_port %>
dir <%= @working_dir %>
daemonize <%= @daemonize ? 'yes' : 'no' %>
pidfile <%= @pid_file %>
26 changes: 26 additions & 0 deletions templates/sentinel/redis-sentinel.conf_master.epp
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<%- |
$master_name,
$redis_host,
$redis_port,
$quorum,
$down_after,
$parallel_sync,
$failover_timeout,
$auth_pass,
$notification_script,
$client_reconfig_script,
| -%>

sentinel monitor <%= $master_name %> <%= $redis_host %> <%= $redis_port %> <%= $quorum %>
sentinel down-after-milliseconds <%= $master_name %> <%= $down_after %>
sentinel parallel-syncs <%= $master_name %> <%= $parallel_sync %>
sentinel failover-timeout <%= $master_name %> <%= $failover_timeout %>
<% if $auth_pass { -%>
sentinel auth-pass <%= $master_name %> <%= $auth_pass %>
<% } -%>
<% if $notification_script { -%>
sentinel notification-script <%= $master_name %> <%= $notification_script %>
<% } -%>
<% if $client_reconfig_script { -%>
sentinel client-reconfig-script <%= $master_name %> <%= $client_reconfig_script %>
<% } -%>
11 changes: 11 additions & 0 deletions types/mastername.pp
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
type Redis::MasterName = Hash[String, Struct[{
redis_host => Stdlib::Host,
redis_port => Stdlib::Port,
quorum => Integer[1],
down_after => Integer[1],
parallel_sync => Integer[0],
failover_timeout => Integer[1],
Optional[auth_pass] => String,
Optional[notification_script] => Stdlib::Absolutepath,
Optional[client_reconfig_script] => Stdlib::Absolutepath,}
],1]