Monday, 8 May 2017

Configure ELK using dockers

I would wish to keep services in three different containers and will try to link to each containers to access ELK stack. 
Installation is very easy .. we will pull docker images and run the containers.

Elasticsearch: 

$sudo docker run --name elasticsearch -d -p 9200:9200 -p 9300:9300 elasticsearch
9c7d52445691015e21a7007e35aca935b9a0dcbbd9560f170cc07c8adc08ae63

$ sudo docker ps -l
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                            NAMES
9c7d52445691        elasticsearch       "/docker-entrypoint.s"   30 seconds ago      Up 30 seconds       0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   elasticsearch
$

test your configurations 

{
  "name" : "PIvLNU_",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Rx7oxSxESvqo-8GNXkDzCA",
  "version" : {
    "number" : "5.3.1",
    "build_hash" : "5f9cf58",
    "build_date" : "2017-04-17T15:52:53.846Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.2"
  },
  "tagline" : "You Know, for Search"
}

Kibana: visualization tool which connects to elasticsearch

since elasticsearch is already running, we need to point kibana container to elasticsearch container.  

$sudo docker run --name kibana -d -p 5601:5601 --link elasticsearch:elasticsearch kibana
2090b8df39a44016e401e8b2fb4e2d79f4d674e9f02ae5794f1ed484fb28913e

$ sudo docker ps -l
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
2090b8df39a4        kibana              "/docker-entrypoint.s"   7 seconds ago       Up 4 seconds        0.0.0.0:5601->5601/tcp   kibana

<script>var hashRoute = '/app/kibana';
var defaultRoute = '/app/kibana';

var hash = window.location.hash;
if (hash.length) {
  window.location = hashRoute + hash;
} else {
  window.location = defaultRoute;
}</script>[sunlnx@fedora ~]$

Point your browser to http://localhost:5601 which would re-direct to kibana default index page..



Logstash:

we would try to take standard input and would it has to be reflected on the kibana dashboard.
create a configuration file for syslog and would start container. 

$cd $PWD/logstash/
$cat logstash.conf

# input from keyboard
input{
stdin {}
}

#output to elasticcontainer
output {
elasticsearch { hosts => ["elasticsearch:9200"] }
}

we are referencing elasticsearch from container and we will link these two containers together. 

$sudo docker run -it --rm --name logstash --link elasticsearch:elasticsearch -v $PWD:/config logstash -f /config/logstash.conf

Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
10:41:27.928 [main] INFO  logstash.setting.writabledirectory - Creating directory {:setting=>"path.queue", :path=>"/var/lib/logstash/queue"}
10:41:28.003 [LogStash::Runner] INFO  logstash.agent - No persistent UUID file found. Generating new UUID {:uuid=>"b359fa87-aef2-4cb1-a533-84767399a0f7", :path=>"/var/lib/logstash/uuid"}
10:41:29.340 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Elasticsearch pool URLs u
.
.
<snip>
this is test1
this is test2
this is test3


logstash would trigger and pushed all messages to elasticsearch, where elasticsearch would created an index page. refresh kibana dashboard, you would now see an index page, click on 'create' and proceed next.

Click on 'discover', you would now see all the messages being taken from keyboard.





Example 2: 

you can also create your own file for port forward TCP and map to your localhost and try to see telnet. you must be able to see the logs in the kibana dashboard. 

$cat portfw.conf

input{
tcp {
   port => 9500
}
}

output {
elasticsearch { hosts => ["elasticsearch:9200"] }
}

$telnet localhost 9500
<type your message> 

I hope you would find this easy for you to configure ELK stack. you could find more reference on the logstash configuration examples from "https://www.elastic.co/guide/en/logstash/current/config-examples.html

Thanks 

Tuesday, 2 May 2017

​Change docker installation directory

If you had only / partition getting filled up with docker images or containers, and you wish to move the docker directory to other location below steps can help you ..

OS: CentOS

Change your default storage base directory for docker(containers and images) from file /etc/sysconfig/docker.

# grep other_args /etc/sysconfig/docker
other_args="-g /your_desired_directory"
#

steps to be taken note while moving from one location to other location: 

1. stop your running containers on the docker 
#docker ps 
#docker stop <container_names>

2. stop docker service 
#service docker stop or systemctl stop docker.service

Double check and confirm docker service stopped. 

3. make sure you backup your current /var/lib/docker before making any changes. 

#tar -cvf var_lib_docker-backup-$(date +%s).tar.gz /var/lib/docker/

4. Move your directory to your desired location 
#mv /var/lib/docker /your_desired_directory

5. Create a symlink to your new diretory
# ln -s /your_desired_directory /var/lib/docker

6. start your docker service
#service docker start or systemctl start docker.service

7. start your containers
# docker ps -a
# docker start <container_names>

Thank you

Sunday, 9 April 2017

Configure your printer on Raspberry Pi

Uses were getting printout from one of locally configured desktop machine which was old running on Ubuntu unfortunately had hardware issues and finally gave a thought for it to RIP. :)
Later, thought about the low cost and tried using Raspberry Pi which acts like a print server, after installing drivers it was still not able to detect printer. I have prepared this tutorial on how to configure print services in Raspberry Pi.

What's needed ?

- 1 Raspberry Pi 3 Model B Board installed with Raspbian OS
- 1 USB based printer connected to Pi board.

This is how it looks like :



Let's start :

Update your repository
# sudo apt-get update

First, in order to link your printer with Raspberry Pi, install CUPS. 
# sudo apt-get install cups 

Once the installation is completed, add usergroup that has access to printer queue. usergroup by default would be 'lpadmin' Since by default user for Rasbian user would be 'pi' add it.
#sudo usermod -a -G lpadmin pi

Configure to enable remote editing for CUPS, rest all can be completed via web browser by pointing your http://localhost:port 
the one in "Green" are to be added and in "red" to be commented. 

#sudo vim /etc/cups/cupsd.conf
# Only listen for connections from the local machine.
#Listen localhost:631 { Comment this line and add below line }
Port 631
Listen /var/run/cups/cups.sock

# Restrict access to the server...
<Location />
  Order allow,deny
  Allow @local
</Location>

# Restrict access to the admin pages...
<Location /admin>
  Order allow,deny
  Allow @local
</Location>

# Restrict access to configuration files...
<Location /admin/conf>
  AuthType Default
  Require user @SYSTEM
  Order allow,deny
  Allow @local
</Location>

Restart your CUPS service
#sudo /etc/init.d/cups restart

In your Raspberry Pi's browser point to http://localhost:631 and click on "Adding Printers and Classes"


Click on "Add Printer" in Administration Panel, select your detected printer and continue and if you get warnings about site certificate ignore it, on which it prompts for username and password of the account you added to 'lpadmin' group earlier in this post.

After logging in, you'll be presented with a list of discovered printers(local or networked). Select the printer you wish to add to your "Pi" 



After continue, you will be prompted to select specific driver you want for your printer. scroll until you see a model number that matches yours. 


​The last configuration step is to Set Default Options and Congrats you have added your system to CUPS. 

Try to display installed printers
#lpstat -a 

Once all are fine, you could issue a print command. 

Thank you. 

Sunday, 26 March 2017

Kerberos Authentication

​Since I had already explained in the past on the mechanism about kerberos, I would try to keep this as much simple as I can.


Go through the below scanned pic, in short have written few notes .. 


Principal Name and key are specified to the client, so clients sends principal name and request for TGT to KDC. 

KDC generates session key(SK) and TGT containing copy of session key, uses TGS to encrypt TGT. Principal key used to encrypt [ Encrypted GT and copy of session key], Client Decrypts using its principal key to extract session key and encrypted TGT.

When client wants to use any service(SSH/NFS..etc) to obtain access for local or remote system( hereafter referred as service provider), it will use session key to encrypt TGT, clients IP addr, time stamp, and SR and sends to KDC

KDC uses its session keys and TGS keys to extract IP addr, time stamp allowing itself to validate client, on successful generates service session key(SSK) and SR containing IP addr+time stamp+copy of SSK and encrypts using service key for SR. 
SK to encrypt both E(SR) and copy of SSK.

Client uses its copy of SK to extract E(E(SR)+SSK)

Client sends the E(SR) to service provider along with E[ Principal name + Time stamp] with E(SSK) 

Service provider uses SK to extract SR which it retrieves SSK to decrypt clients E( Time stamp + Principal Name) 
Once its successful, service provider grants access to its host system.

How to implement kerberos: 

Thanks.

Friday, 9 December 2016

Security auditing tool - Lynis

Have heard about the tool in the past, but hadn't given any try on this... was very simple to go through and here are very few lines on the post...

Ensure you have git client installed on your system we shall clone from github.com

​# cd lynis
# ./lynis audit system 

performs local security scan and will capture all the details in the log file(/var/log/lynis.log)

Then how audit is different from lynis ?

auditd is daemon to track events(like if your /etc/passwd or /etc/shadow file) being changed where as lynis could track file permission etc not the contents in the file. 

  Lynis security scan details:

  Hardening index : 64 [############        ]
  Tests performed : 206
  Plugins enabled : 2

You could explore more on this tool using ./lynis help, anyway would suggest you to give a try

Wednesday, 7 December 2016

Installing Vagrant VM with Oracle Virtual Box

You could have a development environment that is identical to the production environment locally and you can share all your development. Once you or someone else creates a single Vagrantfile, just need to vagrant up and everything is installed and configured for you to work. 

If you are system admin/operation engineer, vagrant gives you a disposable environment for developing, testing infrastructure management scripts like shell scripts, check cookbooks, puppet modules etc 

we shall see how this can be configured and how to work on ... 

You could download vagrant software based on your operating system from https://www.vagrantup.com/downloads.html and install.

I would create a directory called vagrant and would initialize. 

$mkdir ~/workspace/vagrant/centos

$ vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

$ ls
Vagrantfile

Download Oracle virtual box, and since you need to download linux environment which can be from http://www.vagrantbox.es/
I shall download centos 6.7 minimal along with puppet.

we need to remove Vagrant file if we already incase created in the directory.

$ vagrant init vagrant-centos-6.7
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

you would add this box to the provider 'virtualbox' . 

==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'vagrant-centos-6.7' (v0) for provider:
    box:
==> box: Successfully added box 'vagrant-centos-6.7' (v0) for 'virtualbox'!

You can make your setting as you wish in the Vagrant file as its self explanatory , few of my changes which I wished to make 

$ vim Vagrant 

 config.vm.boot_timeout = 60
 config.vm.network "forwarded_port", guest: 80, host: 8080

 config.vm.network "private_network", ip: "192.168.122.15"

we would start the vagrant box which would take some time to bring up the machine..
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 80 (guest) => 8080 (host) (adapter 1)
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Remote connection disconnect. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default:
    default: Guest Additions Version: 4.3.30
    default: VirtualBox Version: 5.0
==> default: Configuring and enabling network interfaces...
==> default: Mounting shared folders...
    default: /vagrant => D:/HashiCorp/workspace/vagrant/centos

Try to login to box using 'vagrant ssh' 

$ vagrant ssh
Last login: Tue Dec  6 16:34:33 2016
[vagrant@localhost ~]$ uptime
 16:38:27 up 1 min,  1 user,  load average: 0.02, 0.02, 0.00
[vagrant@localhost ~]$

[vagrant@localhost ~]$ sudo yum install httpd
Loaded plugins: fastestmirror
Setting up Install Process
Determining fastest mirrors
epel/metalink                                     
.
.

Installed:
  httpd.x86_64 0:2.2.15-55.el6.centos.2

Dependency Installed:
  apr.x86_64 0:1.3.9-5.el6_2                  apr-util.x86_64 0:1.3.9-3.el6_0.1
  apr-util-ldap.x86_64 0:1.3.9-3.el6_0.1      httpd-tools.x86_64 0:2.2.15-55.el6.centos.2
  mailcap.noarch 0:2.1.31-2.el6

Complete!

Install your httpd web server and create your own default Apache webpage to display. 

[vagrant@localhost ~]$ sudo service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
                                                           [  OK  ]
[vagrant@localhost ~]$
[vagrant@localhost ~]$ sudo cat /var/www/html/index.html
Web server is running but no content has been added yet !
Default webpage for this server
[vagrant@localhost ~]$

Point your IP address in the browser and you could see your web applications. this is only a simple configurations as explained using vagrant. 

On your virtual box, below is what you could see while it's running.



More information about this can be looked using 'help' menu .. I would leave this here for the reader to know more .. 
$ vagrant -h
$ vagrant list-commands

Shall try to re-package CentOS6.8(Minimal) and shall explore more on in coming articles..

Thursday, 3 November 2016

git branching and merge for puppetmaster configs modules and manifests - 4/4


As earlier discussed in the post, http://sunlnx.blogspot.in/2016/11/puppet-manifests-and-modules-in.html we shall add version controlling on /etc/puppet and since its important not to distub the main configuration, we shall create a branch out of it and once its all working fine, we shall merge it with master. 

what and why is branching ?

Branching is a process of creating a new pointer that allows you to branch off the code and work on the same code within the safe environment where you are free to mess up and you can discard those changes and you could merge if you feel its correct. git offers that we could switch code from one branch to other and also by merging you don't have any duplicate code. 

As earlier described in article(http://sunlnx.blogspot.in/2015/07/install-and-configure-git-centos-7.html), make sure you install/configure your git repository.

[root@puppetmaster puppet]# git branch
* master
[root@puppetmaster puppet]# git branch puppet-testing
[root@puppetmaster puppet]# git branch
* master
  puppet-testing
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git status
# On branch master
nothing to commit (working directory clean)
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git checkout puppet-testing
Switched to branch 'puppet-testing'
[root@puppetmaster puppet]# git status
# On branch puppet-testing
nothing to commit (working directory clean)
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git branch
  master
* puppet-testing
[root@puppetmaster puppet]#

Now you can do anything you wish to the branch, like deleting, modifying ...

[root@puppetmaster puppet]# cat > git-branching.test
this is just an file created in puppet-testing branch.
we shall see how to merge this into 'master' branch.
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# ls -l git-branching.test
-rw-r--r-- 1 root root 108 Nov  3 04:58 git-branching.test
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git add *
[root@puppetmaster puppet]# git commit -m "Added git-branching to repos"
[puppet-testing 386d918] Added git-branching to repos
 1 files changed, 2 insertions(+), 0 deletions(-)
 create mode 100644 git-branching.test
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git push origin puppet-testing
Password:
Counting objects: 4, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 369 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local objects.
 * [new branch]      puppet-testing -> puppet-testing
[root@puppetmaster puppet]#

Refresh you repository in github and you can see branch puppet-testing. you could also create a new branch again from 'puppet-testing' and it can go on as much as you can. something great that's more flexible on git.

Now in order to merge your branch, switch to master branch 

[root@puppetmaster puppet]# git checkout master
Switched to branch 'master'
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git merge puppet-testing
Updating 285446b..386d918
Fast-forward
 git-branching.test |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
 create mode 100644 git-branching.test
[root@puppetmaster puppet]# git branch -d puppet-testing
Deleted branch puppet-testing (was 386d918).
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git push origin master
Password:
Total 0 (delta 0), reused 0 (delta 0)
   285446b..386d918  master -> master
[root@puppetmaster puppet]#

[root@puppetmaster puppet]# git push origin --delete puppet-testing
Password:
 - [deleted]         puppet-testing
[root@puppetmaster puppet]#

You can now refresh your github.com and see there won't be branch available. checkout in your master branch the file was merged to master..

Thanks for re-sharing !