Apache solr replication step by step

Need for Replication :

• When there is a large search volume that cannot be handled by a single machine, so you need to distribute searches across multiple read-only copies of the index.
• If there is a high volume of indexing which consumes machine resources and reduces search performance on the indexing machine, so you need to separate indexing and searching.
• When we want to make a backup of the index

MASTER-SLAVE

• Distributes complete copies of a master index to one or more slave servers. 
• The master server continues to manage updates to the index. 
• All querying is handled by the slaves. 
• This enables Solr to scale to provide adequate responsiveness to queries against large search volumes.

Replication Terminology:

Index

A Lucene index is a directory of files. These files make up the searchable and returnable data of a Solr Core.

Distribution

The copying of an index from the master server to all slaves. 

Inserts and Deletes

• As inserts and deletes occur in the index, the directory remains unchanged. Documents are always inserted into newly created files. 
• Documents that are deleted are not removed from the files.
• They are flagged in the file, deletable and are not removed from the files until the index is optimized.

Master and Slave

• A Solr replication master is a single node which receives all updates initially and keeps everything organized.
• Solr replication slave nodes receive no updates directly, instead all changes (such as inserts, updates, deletes, etc.) are made against the single master node. 
• Changes made on the master are distributed to all the slave nodes which service all query requests from the clients.

Repeater

A node that acts as both a master and a slave.

Optimization

• A process that compacts the index and merges segments in order to improve query performance.
• Optimization should only be run on the master nodes. An optimized index may give query performance gains compared to an index that has become fragmented over a period of time with many updates. 
• Distributing an optimized index requires a much longer time than the distribution of new segments to an un-optimized index

Snapshot

A directory containing hard links to the data files of an index. Snapshots are distributed from the master nodes when the slaves pull them, "smart copying" any segments the slave node does not have in snapshot directory that contains the hard links to the most recent index data files.

Configuring the Replication RequestHandler on a Master Server:

commit-Triggers replication whenever a commit is performed on the master index.

optimize-Triggers replication whenever the master index is optimized.

startup-Triggers replication whenever the master index starts up.

MASTER SLAVE CONFIGURATION

MASTER

The configuration of the master are configured in the below file.

vim /var/solr/data/fortis/conf/solrconfig.xml

vim /var/solr/data/fortis/core.properties

Make the below changes in the core.properties file 

enable.master=true

enable.slave=false

SLAVE

To create a new collection use the below command

/usr/local/solr-6.6.2/bin/solr create -c prod

The configuration of the master are configured in the below file.

vim /var/solr/data/fortis/conf/solrconfig.xml

Make the below changes in the core.properties file 

vim /var/solr/data/fortis/core.properties

enable.master=false

enable.slave=true

Now we have setup the master-slave replication in the apache solr.

Read More

User creation in MongoDB

                   Security is a foremost criteria in the databases. Proper authentication in the databases must be enabled to ensure high security.Users in the database paves a way for the better authentication in databases.In this post we are going to see how to create the users once you are done with the MongoDb installation.Its necesary to create the required range of the users.Each users have their own roles that they can perform.The following are the some users that are very much necessary in the mongodb databases .

Administrator user:

                     This user can grant themselves privileges in excess of their current privileges and even can grant themselves all privileges, even though the role does not explicitly authorize privileges beyond user administration.It is also a typical superuser in the database.

use admin

db.createUser(
 {
   user: "UserAdmin",
   pwd: "password",
   roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
 }
)

Root user:

                          It provides access to the operations and all the resources of the readWriteAnyDatabase, dbAdminAnyDatabase, userAdminAnyDatabase and clusterAdmin roles combined also this user does not include any access to collections that begin with the system. prefix.

use admin

db.createUser(
{
user: "root",
pwd: "password",
roles: [ "root" ]
}
)

Read/Write user:

This user provides the read and write permissions for a particular database that is specified.

use admin

db.createUser(

  {

    user: "rw_user",

    pwd: "password",

    roles:

      [

        { role: "readWrite",db:"test_db"}

      ]

  }

)

Read user:

This user provides only the read permissions for a particular database that is specified.

use admin

db.createUser(

  {

    user: "read_user",

    pwd: "password",

    roles:

      [

        { role: "read",db:"test_db"}

      ]

  }

)

Write user:

This user provides only the write permissions for a particular database that is specified.

use admin

db.createUser(

  {

    user: "write_user",

    pwd: "password",

    roles:

      [

        { role: "write",db:"test_db"}

      ]

  }

)

Backup User:

This user is used to perform only the backup operations in the database.So in case you have single instance its enough to have the following role:

use admin

db.createUser(

  {

    user: "backup",

    pwd: "password",

    roles:

      [

        { role: "dbAdmin",db:"admin"}

      ]

  }

)

Incase you maintain any shard or replica you can have the below role:

use admin

db.createUser(

  {

    user: "backup",

    pwd: "password",

    roles:

      [

        { role: "clusterdbAdmin",db:"admin"}

      ]

  }

)

Check user privileges:

Its very much necessary to see what are all the privileges that are given to a particular user,to find them use the below query:

db.runCommand(

              {

                usersInfo:  { user: "root", db: "admin" },

                showPrivileges: true

              }

)

                    You can also create your own users similarly mentioned above if you have more ideas about the roles needed.To know more about the mongodb roles check this link.

Read More