How To Use Sidekiq PRO and OAuth to Monitor the Workers of Multiple Rails Applications On Heroku

May 17, 2016

Using Sidekiq PRO and Github OAuth to Monitor Production Sidekiq Workers on Different Rails Applications

Want an easy way to do the above title? You don’t want to create another User type table on your Rails application just to track authorized users to monitor each Rails application running Sidekiq workers. So let’s create a standalone Rails application service to do just that.

Install Sidekiq PRO

The below is the regular sidekiq gem (for example purposes). You actually need to buy a license from the author of Sidekiq and he will give you a different gem to use. So the below syntax is just a placeholder to give you an idea of what is going on.

gem 'sidekiq'
gem 'omniauth'
gem 'omniauth-github'
gem 'octokit'
gem 'sinatra', require: nil #for Sidekiq Web UI

Install Devise and Generate a User

Follow the standard devise installation.

rails generate devise:install

rails generate devise User

Add an “organization” database column for the User of type string.

rails g migration AddOrganizationToUser

The User model will look like this:

class User &lt; ActiveRecord::Base

# Include default devise modules. Others available are:

# :confirmable, :lockable, :timeoutable and :omniauthable

devise :database_authenticatable, :registerable,
:recoverable, :rememberable, :trackable, :validatable, :omniauthable</code>

validates_presence_of :organization

def self.from_omniauth(auth)
  where(provider: auth.provider, uid: auth.uid, organization: get_organization(auth)).first_or_create! do |user|
    user.email = auth.info.email
    user.password = Devise.friendly_token[0,20] #user.name = auth.info.name # assuming the user model has a name
    #user.image = auth.info.image # assuming the user model has an image
  end
end

def is_part_of_organization?(org)
  organization == org
end

private

  def self.get_organization(auth)
    octo_client = Octokit::Client.new(access_token: auth.credentials.token)
    octo_client.organization_member?("LootCrate", auth.extra.raw_info.login) ? "LootCrate" : nil
  end
end

Add the correct route and authentication block

In config/routes.rb:

devise_for :users, controllers: { omniauth_callbacks: "users/omniauth_callbacks" }
  resource :users, only: [:new]

  require 'sidekiq/pro/web'

  authenticate :user, lambda { |u| u.is_part_of_organization?("MyOrganization") } do

  # if adding extra mountable routes, add an underscore (or change the regex in application_controller.rb)

  mount Sidekiq::Pro::Web.with(redis_pool: POOL1), at: "/sidekiq", as: :sidekiq

  mount Sidekiq::Pro::Web.with(redis_pool: POOL2), at: "/sidekiq2", as: :sidekiq2

end

Add an omniauth_callbacks_controller

In controllers/users/omniauth_callbacks_controller:

class Users::OmniauthCallbacksController < Devise::OmniauthCallbacksController
  def github
  # You need to implement the method below in your model (e.g. app/models/user.rb)
    @user = User.from_omniauth(request.env["omniauth.auth"])
    if @user.persisted?
      sign_in_and_redirect @user, :event => :authentication #this will throw if @user is not activated
      set_flash_message(:notice, :success, :kind => "Github") if is_navigational_format?
    else
      session["devise.github_data"] = request.env["omniauth.auth"]
      redirect_to new_user_registration_url
    end
  end

  def failure
    redirect_to root_path
  end

end

Create the correct key and secret on Github

Go to Github.com and create a key and secret. You can do this on your own profile to test it on localhost. The owner of your organization’s repository will have to do it for the production repository.

Steps:

1 - Edit Profile > Applications > Developer Applications

2 - Fill in Homepage URL and Authorization callback URL with the appropriate URLs.

Homepage URL = http://localhost:3000/

Authorization callback URL = http://localhost:3000/users/auth/github/callback

Configure config/initializers correctly

In config/initializers/devise.rb:

config.omniauth :github, ENV['GITHUB_CLIENT_KEY'], ENV['GITHUB_CLIENT_SECRET'], callback_url: "/users/omniauth_callbacks", scope: "user:email,read:org"

In config/initializers/sidekiq.rb

POOL1 = ConnectionPool.new { Redis.new(:url => ENV['REDIS_1']) }
POOL2 = ConnectionPool.new { Redis.new(:url => ENV['REDIS_2']) }

Customize after_sign_in_path_for helper in application_controller.rb

You will notice that below I do something weird with gsub method. This is because somehow devise (or warden or rack, I’m not sure which) turns a path like “/sidekiq2” into “/sidekiq/_2” on initial sign in through github and you end up getting a navigation error.

class ApplicationController &lt; ActionController::Base
  # Prevent CSRF attacks by raising an exception.
  # For APIs, you may want to use :null_session instead.

  protect_from_forgery with: :exception

  def after_sign_in_path_for(resource)
    path = stored_location_for(resource)
    path.gsub(/\/\_/, '\_') #because devise returns '/sidekiq/_webhooks' on routes with underscore (or any character after q)
  end
end

Configure the ENV variables and redis URLs on Heroku

Set GITHUB_CLIENT_KEY and GITHUGB_CLIENT_SECRET to the appropriate values that your production repository owner obtained from Github.

Set REDIS_1 and REDIS_2 to the URLs of the redis databases (in my case, I was using redis2go) of each Heroku Rails application whose sidekiq jobs you want to monitor.

And you’re done! You can now monitor multiple Rails applications on Heroku running Sidekiq workers.