Apache SOLR introduction & installation

In this post, we are going to discuss about Apache Solr and how to install them step by step.

Why Solr?



Lucene:

• Search storage engine
• Solr uses concepts derived from Lucene
• Lucene is widely used in many projects among one of them is solr
• Used to index & search with high performance.
• Solr uses lucene as its backend

Solr:

• Search server
• Document oriented
• Stores data & indexing
• Searches including full-text search, stemming,hit-highlighting, faceted-search etc that cannot done by native databases
• Vertically and horizontally scalable
• Replication for high availability
• Sharding for distributed search
• Performs in memory,grouping,counting,similar products in single shot
• Exposed over HTTP,REST like api.
• The DataImportHandler provides a configuration driven way to import data from relational databases or XML files, into Solr in both “full import” and “incremental delta import” mode.

INVERTED INDEX:

• Searches document by unique word
• Similar to the index back of back

D1- I like Apache services.
D2- They include all kind of database & services support.
D3- I would recommend their services to my clients too.

Working:

• Define a schema. 
• Deploy Solr.
• Feed Solr documents for which your users will search.
• Expose search functionality in your application.

Solr schema:

• No schema
• Has index that contain docs
• Fields are used to index,search & store

Define a schema

The schema tells Solr about the contents of documents it will be indexing. In the online store example, the schema would define fields for the product name, description, price, manufacturer, and so on. 

Defining fields:

Indexing:

Indexing is a technique of adding Document’s content to Solr Index so that we can search them easily. Apache Solr uses Apache Lucene Inverted Index technique to Index it’s documents. That’s why Solr provides very fast searching feature.

Field analyzers: (Analyzer=tokenizer+filters)

Used both during ingestion, when a document is indexed, and at query time.Analyzers may be a single class or series of tokenizer and filter classes.

Alternative words-finish,complete

misspelled-google,gogle

Tokenizers:

Break field data into lexical units, or tokens.

Pre- Stripping html tags

Post-Stemming(replace) tables as collection

         Stop word filter (the,is,and)

Filters:(used for indexing)

• Examine a stream of tokens and keep them, transform or discard them, or create new ones. 
• Tokenizers and filters may be combined to form pipelines, or chains, where the output of one is input to the next. 
• Such a sequence of tokenizers and filters is called an analyzer and the resulting output of an analyzer is used to match query results or build indices.

eg)ram,RAM,Ram

Deploying Solr

Pre requisites:

yum update

yum install java-1.8.0-openjdk.x86_64

java -version

Installation:

wget http://apache.org/dist/lucene/solr/6.6.1/solr-6.6.1.tgz

tar zxvf solr-6.6.1.tgz
cp /opt/solr-6.6.1/bin/install_solr_service.sh .
rm -rf solr-6.6.1
./install_solr_service.sh solr-6.6.1.tgz
ps -ef | grep solr

Feed Solr documents for which your users will search Creating a project(core): A Core is an Index of texts and fields available in all documents. One Solr Instance can contain one or more Solr Cores. /opt/solr-6.6.1/bin/solr create -c jerwin Create new document : We can create a new document to the core with 3 fields and value using the below command on the terminal using curl: curl http://localhost:8983/solr/jerwin/update -d ' [ {"id" : "db1", "company_name" : "Mafiree", "location" : "Nagercoil" } ]'

View:

The inserted data can be viewed with the id specified.

curl http://localhost:8983/solr/jerwin/get?id=db1

Output can be viewed in the browser:

Hope this gives you simple introduction about SOLR,let me know if you have any concerns via comments.

Read More

Install MongoDB using yum on Linux(CentOS)

                MongoDB falls into NoSQL category of database also it is a document database that provides high performance, high availability, and easy scalability.Few of its advantages are high availability and scalability.In this post we are going to see how to install the mongodb using yum on linux.

Step 1:Create a repo file that is necessary to download the mongodb.

vim /etc/yum.repos.d/mongodb.repo

Step 2:Mongodb has two types of packages for 32-bit and 64-bit systems.

For 32-bit systems type the below information in repo file and save.

[mongodb-org-3.0]

name=MongoDB 3.0 Repository

baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686/

gpgcheck=0

enabled=1

         In the first line of the file specify you own necessary version of the mongodb to be downloaded.Here i have used version 3.0.

For 64-bit systems type the below information in repo file and save.

[mongodb-org-3.0]
name=MongoDB 3.0 Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
gpgcheck=0
enabled=1

         In the first line of the file specify you own necessary version of the mongodb to be downloaded.Here i have used version 3.0.

Note:It’s always better to download 64 bit for production rather than 32 bit because 32-bit MongoDB processes are limited to about 2 gb of data.But the 64 bit doesn't have that limitation. For development both versions can be used.

Step 3:Mongodb has a set of packages as below:

mongodb-org-server-3.0.4           //Mongodb server

mongodb-org-shell-3.0.4            //Mongo Shell

mongodb-org-mongos-3.0.4           //Mongo Shard

mongodb-org-tools-3.0.4            //Mongo Tools

And it contains all the above package together as a single package as below:

mongodb-org-3.0.4

To install all the packages issue the below command:

sudo yum install mongodb-org

To install mongodb server alone issue the below command:

sudo yum install mongodb-org-server-3.0.4

To install mongo shell alone issue the below command:

sudo yum install mongodb-org-shell-3.0.4

To install mongo shard alone issue the below command:

sudo yum install mongodb-org-mongos-3.0.4

To install mongo tools alone issue the below command:

sudo yum install mongodb-org-tools-3.0.4

Step 3:Start the installed mongodb server using the below command:

sudo service mongod start

The server is now started and to login to the mongo shell issue the below command:

mongo

To check the current status of mongod issue the below command:

sudo service mongod status

To stop the running mongod server use the below command:

sudo service mongod stop

It’s finished.We have installed the mongodb and now you can start exploring :)

Read More

Install MongoDB on Ubuntu

             MongoDB is a cross-platform, document oriented database that provides,high performance, high availability, and easy scalability.This guide helps to install the mongodb on ubuntu.

Step 1: Search for the mongodb package avaliable but initially it may not be present

sudo apt-cache search mongodb

Step 2: Add the package to /etc/apt/sources.list.d because its the the latest stable MongoDB version

vim /etc/apt/sources.list.d/mongodb.list

Add the following link into the mongodb.list file

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

Step 3: Import the GPG key

apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Step 4 : Update the debian package using the command:

apt-get update

Step 5: Now perform search to view the package

apt-cache search mongodb

Step 6: Now the package will be present so install it using the below command:

apt-get install mongodb-10gen

Step 7: Check mongodb has started using

ps -ef | grep mongo

ls -ls /usr/bin | grep mongo

Step 8: The mongodb control script is generated at /etc/init.d/mongodb,the MongoDB config file is at /etc/mongodb.conf add the below details:

 verbose = true
dbpath = /data/db
logpath = /var/log/mongodb.log
logappend = true
port = 27017

Step 9: Start the mongodb as a service

service mongodb start

Step 10 :Check mongodb service has started using

ps -ef | grep mongodb

Step 11 : To stop/restart perform the below commands:

service mongodb stop

service mongodb restart

Thus the installation has been very simple in ubuntu.

Read More

Install MongoDB binary on linux

                  

MongoDB is a cross-platform, document-oriented database that provides, high performance, high availability, and easy scalability. MongoDB works on the concept of collection and document. It is also one of the leading NoSQL databases. It’s easy to install MongoDB binary file within five minutes if you follow the steps mentioned below. Here we go..!!

Step 1: Download the latest .tar.gz file from  this link

Its always better to download 64 bit for production rather than 32 bit because 32-bit MongoDB processes are limited to about 2 GB of data. But the 64 bit doesn't have that limitation. For development, both versions can be used.

Step 2: Create a source directory and place the downloaded file in the source directory.For example (/usr/local)

Step 3: Now extract the downloaded file using the below command:

tar zxf mongodb-linux-i686-1.6.0.tar.gz

Step 4: Now rename the extracted(mongodb-linux-x86 64-2.6.3) file into MongoDB.

mv mongodb-linux-x86 64-2.6.3 ./mongodb

Step 5: Create a data directory using the below command

mkdir /data/db

Step 6: Create a user mongo_user using the following command

useradd mongod

                 This user can have all the privileges to access the MongoDB database and directories.

Step 7: Change the ownership of the files in the source and data directory using the following command

chown -R mongod.mongod /usr/local/

chown -R mongod.mongod /data/db/

                     Changing the ownership gives the complete access for the users to access the source and the data directory.

Step 8: Create a configuration file in any directory say vim /etc/mongod.conf

Now add the following details as shown below:

verbose = true
dbpath = /data/db
logpath = /var/log/mongodb.log
logappend = true
port = 27017

                    Verbose is an option that provides additional details what the computer is doing and what drivers and software it is loading during startup. Dbpath shows the data directory path. Logpath shows the path for the log file.Log append will append the log file. Port will specify which port the mongo shell will connect.

Step 9: Move to the bin folder in the source directory and now start the mongo server by the following command

./mongod --config /etc/mongod.conf

Step 10: Within the bin folder in the source directory start the mongo client by the following command

./mongo --port 27017

The interactive MongoDB prompt appears. That's it now you're done with the MongoDB binary installation. Time to play with MongoDB :)

Read More