tag:blogger.com,1999:blog-46525598977116048392024-03-18T13:16:51.293+05:30milearningEngineer who's on a journey learning technical stuff and sharing my experiences ! milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.comBlogger128125tag:blogger.com,1999:blog-4652559897711604839.post-72463557310946550622021-02-23T14:16:00.000+05:302021-02-23T14:17:00.505+05:30Point yum to Centos Repository<div dir="ltr"><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000">Point Centos repository to download packages. </div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"><br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000">In here you can just add these to your /etc/yum.repos.d/centos.repo file and paste these contents. <br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"><div class="gmail_default"></div></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"><br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000">[base]<br>name=CentOS $releasever – Base<br>baseurl=<a href="http://mirror.centos.org/centos/7/os/$basearch/">http://mirror.centos.org/centos/7/os/$basearch/</a><br>gpgcheck=0<br>enabled=1<br><br>[updates]<br>name=CentOS $releasever – Updates<br>baseurl=<a href="http://mirror.centos.org/centos/7/updates/$basearch/">http://mirror.centos.org/centos/7/updates/$basearch/</a><br>gpgcheck=0<br>enabled=1<br><br>[extras]<br>name=CentOS $releasever – Extras<br>baseurl=<a href="http://mirror.centos.org/centos/7/extras/$basearch/">http://mirror.centos.org/centos/7/extras/$basearch/</a><br>gpgcheck=0<br>enabled=1<br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"><br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000">Install epel repo,</div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"><br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000">rpm -Uvh <a href="https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm">https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm</a><br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"><br></div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"># yum clean all</div><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;color:#000000"># yum list installed</div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">Thanks,</font></div><div><font face="trebuchet ms, sans-serif">Sunil</font></div></div></div></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-83727258850847853982020-11-14T17:39:00.000+05:302020-11-14T17:40:38.436+05:30Elasticsearch/Kubernetes/Logstash(ELK) - Part 5<div dir="ltr"><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)"></span><span class="gmail_default" style="color:rgb(0,0,0)"></span><font size="4">File Beats</font><br><br>Whether you're collecting from security devices, cloud, containers, hosts, or OT, Filebeat helps you keep the simple things simple by offering a lightweight way to forward and centralize logs and files.<span class="gmail_default" style="color:rgb(0,0,0)">(Easlticsearch)</span></font><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)"><br></span></font></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)">Reference: </span></font></div><div><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/beats/filebeat" target="_blank">https://www.elastic.co/beats/filebeat</a><span class="gmail_default" style="color:rgb(0,0,0)"></span><span class="gmail_default" style="color:rgb(0,0,0)"><br></span></font></div><div><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/beats/filebeat/current/index.html" target="_blank">https://www.elastic.co/guide/en/beats/filebeat/current/index.html</a></font></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)"></span> </font></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default">How does the file beat work?</span></font></div><div><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html" target="_blank">https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html</a><span class="gmail_default"><br></span></font></div><div><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/beats/filebeat/6.3/how-filebeat-works.html" target="_blank">https://www.elastic.co/guide/en/beats/filebeat/6.3/how-filebeat-works.html</a></font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)">How to install filebeat?</span></font></div><div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">From your kibana console -> Logging -> Apache Metrics -> Download the filebeats and configure.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/beats/filebeat/6.3/logstash-output.html" target="_blank">https://www.elastic.co/guide/en/beats/filebeat/6.3/logstash-output.html</a><br></font></div><font face="trebuchet ms, sans-serif"><br></font></div><div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">I have modified the logstash file so that input would be using the filebeats ..</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">cat apache-filebeat.conf</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><font face="trebuchet ms, sans-serif"><font color="#741b47">input<br>{<br> beats {<br> port => 5044<br> }<br>}<br><br>filter<br>{<br> grok{<br> match => {<br> "message" => "%{COMBINEDAPACHELOG}"<br> }<br> }<br> mutate{<br> convert => { "bytes" => "integer" }<br> }<br> date {<br> match => [ "timestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]<br> locale => en<br> remove_field => "timestamp"<br> }<br> geoip {<br> source => "clientip"<br> }<br> useragent {<br> source => "agent"<br> target => "useragent"<br> }<br>}<br><br>output<br>{<br> stdout {<br> codec => dots<br> }<br> elasticsearch {<br> hosts => ["<a href="http://localhost:9200" target="_blank">http://localhost:9200</a>"]<br> index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"<br> }<br>}</font></font></div><div><font face="trebuchet ms, sans-serif"><font color="#741b47"><br></font></font><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Start the logstash server from a new terminal, search for the logs which would listen to the beats..</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default"><font face="trebuchet ms, sans-serif"><font color="#741b47">C:\elk\logstash>bin\logstash.bat -f C:\elk\data\apache-filebeat.conf<br>Sending Logstash's logs to C:/elk/logstash/logs which is now configured via log4j2.properties<br>[2020-11-14T17:18:12,918][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified<br>[2020-11-14T17:18:13,368][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.2"}<br>[2020-11-14T17:18:16,372][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}<br>[2020-11-14T17:18:16,784][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[<a href="http://localhost:9200/%5D" target="_blank">http://localhost:9200/]</a>}}<br>[2020-11-14T17:18:16,784][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=><a href="http://localhost:9200/" target="_blank">http://localhost:9200/</a>, :path=>"/"}<br>[2020-11-14T17:18:16,971][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"<a href="http://localhost:9200/" target="_blank">http://localhost:9200/</a>"}<br>[2020-11-14T17:18:17,018][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}<br>[2020-11-14T17:18:17,018][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}<br>[2020-11-14T17:18:17,049][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["<a href="http://localhost:9200" target="_blank">http://localhost:9200</a>"]}<br>[2020-11-14T17:18:17,065][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}<br>[2020-11-14T17:18:17,096][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}<br>[2020-11-14T17:18:17,252][INFO ][logstash.filters.geoip ] Using geoip database {:path=>"C:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}</font><br><b><font color="#cc0000">[2020-11-14T17:18:17,811][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"<a href="http://0.0.0.0:5044" target="_blank">0.0.0.0:5044</a>"}</font></b><br><font color="#741b47">[2020-11-14T17:18:17,827][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x7758f1a6 run>"}<br>[2020-11-14T17:18:17,936][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}<br>[2020-11-14T17:18:17,943][INFO ][org.logstash.beats.Server] Starting server on port: 5044<br>[2020-11-14T17:18:18,208][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}</font><br></font></div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)">It's now time to modify the filebeat configuration. </span></font></div><div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><b>config file: filebeat.yml</b></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Change these in the config file..<br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif"><font color="#741b47">filebeat.inputs:<br></font></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif"><font color="#741b47">- type: log<br> enabled: true<br> paths:<br> - C:\elk\data\logs\*</font><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Comment out from elastic search and un-comment logstash</div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif"><font color="#741b47">#----------------------------- Logstash output --------------------------------<br>output.logstash:<br> hosts: ["localhost:5044"]</font><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Save and Quit, start the filebeat server. <br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Open a new terminal and execute below command</div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif"><font color="#0000ff">elk/filebeat>filebeat.exe</font></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyEg4q8LrtE9xQM41gjlArerdJQgI4GgtTfbA1hM2xJwdbcLu8CLFKwriOS6EHL89UoLZE-pA0L39JMBWPwcvpHAhyVYs7CgJwKi7IhAECdr3nulAjodDN29r_RL6YENEEjYzlheuYaeQ/s1600/image-711052.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyEg4q8LrtE9xQM41gjlArerdJQgI4GgtTfbA1hM2xJwdbcLu8CLFKwriOS6EHL89UoLZE-pA0L39JMBWPwcvpHAhyVYs7CgJwKi7IhAECdr3nulAjodDN29r_RL6YENEEjYzlheuYaeQ/s320/image-711052.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6894950710307252802" /></a><br></div></div><br></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)">Go to the Kibana console and check on the Management, you would find all the index building done by the logstash.</span></font></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="color:rgb(0,0,0)"><br></span></font></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCID4OcgdiCNrGBFd8NLktU0mvjTXPU2d0urmDVyta5eV0cSyfllLfgjs6ttUa3KqOJoWtyQXeu8SAv78DtIu7ifNGuX_5uuNGO_9Kw0AgOZaGgcXN1DXg2PxIPDIy_LQn2CXhITKFGjc/s1600/image-712852.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCID4OcgdiCNrGBFd8NLktU0mvjTXPU2d0urmDVyta5eV0cSyfllLfgjs6ttUa3KqOJoWtyQXeu8SAv78DtIu7ifNGuX_5uuNGO_9Kw0AgOZaGgcXN1DXg2PxIPDIy_LQn2CXhITKFGjc/s320/image-712852.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6894950716801174882" /></a><br></div></div><div><br></div><div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">All your data is being sent from the host using filebeat to the logstash server. </div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">That's all for ELK. </div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><br></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-3232595366682782102020-11-14T15:29:00.000+05:302020-11-14T17:40:50.725+05:30Elasticsearch/Kubernetes/Logstash(ELK) - Part 4<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif" size="4"><b>Installations of Logstash with Kibana</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Install the logstash, and start the service. You can test if its working fine using the below command, </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><b><font face="trebuchet ms, sans-serif">bin/logstash -e '{ input stdin{}}' '{ output stdout{}}'</font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">This means whatever you type would be taken as input to logstash and would output the same on the output. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">I would be taking an Apache log file and would create a logstash config file where it would load all the data that has to be searched and loaded as an index into kibana. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Place this file in your elk folder, create another folder as data and place this file over there.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><b>Reference:</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/logstash/current/input-plugins.html">https://www.elastic.co/guide/en/logstash/current/input-plugins.html</a><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/logstash/current/output-plugins.html">https://www.elastic.co/guide/en/logstash/current/output-plugins.html</a><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><a href="https://www.elastic.co/guide/en/logstash/current/filter-plugins.html">https://www.elastic.co/guide/en/logstash/current/filter-plugins.html</a><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif">input<br>{<br> file {<br> path => "C:\elk\data\logs\logs"<br> type => "logs"<br> start_position => "beginning"<br> }<br>}<br><br>filter<br>{<br> grok{<br> match => {<br> "message" => "%{COMBINEDAPACHELOG}"<br> }<br> }<br> mutate{<br> convert => { "bytes" => "integer" }<br> }<br> date {<br> match => [ "timestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]<br> locale => en<br> remove_field => "timestamp"<br> }<br> geoip {<br> source => "clientip"<br> }<br> useragent {<br> source => "agent"<br> target => "useragent"<br> }<br>}<br><br>output<br>{<br> stdout {<br> codec => dots<br> }<br> elasticsearch {<br> }<br>}<br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif">Download the apache log(<a href="https://github.com/elastic/elk-index-size-tests/blob/master/logs.gz">https://github.com/elastic/elk-index-size-tests/blob/master/logs.gz</a>) from whatever source you have mentioned in the above document(C:\elk\data\logs\logs) and start the logstash server. All these logs would be read from the elasticsearch as the output is destined over there. we could see the same from the kibana console.</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><span style="color:rgb(0,0,255);font-family:"trebuchet ms",sans-serif">C:\elk\logstash>bin\logstash.bat -f C:\elk\data\apache.conf</span><br></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" color="#741b47">Sending Logstash's logs to C:/elk/logstash/logs which is now configured via log4j2.properties<br>[2020-11-14T09:41:21,710][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"C:/elk/logstash/data/queue"}<br>[2020-11-14T09:41:21,722][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"C:/elk/logstash/data/dead_letter_queue"}<br>[2020-11-14T09:41:21,880][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified<br>[2020-11-14T09:41:21,927][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"ec39c09a-712e-4d86-a9d8-ab629546e04f", :path=>"C:/elk/logstash/data/uuid"}<br>[2020-11-14T09:41:22,721][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.2"}<br>[2020-11-14T09:41:27,029][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}<br>[2020-11-14T09:41:27,751][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[<a href="http://127.0.0.1:9200/]">http://127.0.0.1:9200/]</a>}}<br>[2020-11-14T09:41:27,767][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=><a href="http://127.0.0.1:9200/">http://127.0.0.1:9200/</a>, :path=>"/"}<br>[2020-11-14T09:41:28,095][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"<a href="http://127.0.0.1:9200/">http://127.0.0.1:9200/</a>"}<br>[2020-11-14T09:41:28,189][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}<br>[2020-11-14T09:41:28,189][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}<br>[2020-11-14T09:41:28,236][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//<a href="http://127.0.0.1">127.0.0.1</a>"]}<br>[2020-11-14T09:41:28,251][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}<br>[2020-11-14T09:41:28,298][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}<br>[2020-11-14T09:41:28,392][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash<br>[2020-11-14T09:41:28,801][INFO ][logstash.filters.geoip ] Using geoip database {:path=>"C:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}<br>[2020-11-14T09:41:30,949][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x58a3a982 run>"}<br>[2020-11-14T09:41:31,074][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}<br>[2020-11-14T09:41:31,851][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}<br>.....................................................................................................<br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" color="#741b47">.</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" color="#741b47">.</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif">Once these dots are completed, you can close this window. You can test the same in the kibana console that you have all the indexes from the logstash..</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif">[ ctrl-c]</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><font color="#741b47">..................................................................[2020-11-14T10:46:36,690][WARN ][logstash.runner ] SIGINT received. Shutting down.<br>[2020-11-14T10:46:38,334][INFO ][logstash.pipeline ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x58a3a982 run>"}<br>Terminate batch job (Y/N)? y</font><br><br>C:\elk\logstash><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><div><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHSY0Kwh_-Ictug2FSZBjOXbNvVz31uEK2H6pf9jglmBhDo63xtPN4TJe9UaymThh73vM1LaJhE4D0Y5tNwU5tlotKZTG6sZN3IVIdj0jqi3CQdSrQqdYAxOhCKhJ55LrE77WU5I3nVtY/s1600/image-707767.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHSY0Kwh_-Ictug2FSZBjOXbNvVz31uEK2H6pf9jglmBhDo63xtPN4TJe9UaymThh73vM1LaJhE4D0Y5tNwU5tlotKZTG6sZN3IVIdj0jqi3CQdSrQqdYAxOhCKhJ55LrE77WU5I3nVtY/s320/image-707767.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6894917193265805506" /></a><br></font></div></div><div class="gmail_default" style=""><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">you can also check from the elasticsearch url</font></div><div><font face="trebuchet ms, sans-serif"><a href="http://localhost:9200/logstash-*/_count">http://localhost:9200/logstash-*/_count</a><br></font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">Now, login to kibana, and go to Discover and create an index pattern using @timestamp (select range from 01-June-2014 to 20-July-2014)and you see the diagram below. Once the datas is loaded you will need to get a playaround, creating your own visuals and those being presented in the Dashboard.</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><div><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSWixqIrEkElk4omcdTC_yF0eFN5bSdWvNJYkGDdhq3jIFzmG5H0yAwSix0Ah63P6K9Nad6J6jkiohbMGVNGgGUvmt-jOkaJlJqZIUJED8dhNmrRB9Zgc4Ox-SlPREEuFANlcW1jm1F2I/s1600/image-709764.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSWixqIrEkElk4omcdTC_yF0eFN5bSdWvNJYkGDdhq3jIFzmG5H0yAwSix0Ah63P6K9Nad6J6jkiohbMGVNGgGUvmt-jOkaJlJqZIUJED8dhNmrRB9Zgc4Ox-SlPREEuFANlcW1jm1F2I/s320/image-709764.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6894917203621257762" /></a><br></font></div></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">We can create metrics of various fields and add into a single dashboard. </font></div><div><font face="trebuchet ms, sans-serif">Created Pie-Charts, Bar Charts, & Geo-location graphs into the dashboard for the total requests during the mentioned time line.</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><div><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq7LEbYb3OsxDtNLU59pbG9GiOReOuEil73sN1SK7hAHhD4YNld3kDyyJdsOaCGmalhZvSNb4lGbxir9RdNuwh70Mw1sI2HpblR_HyWh0hxvYRiboM7bu42BQjtwNx8n_Xer1XcUE2G2A/s1600/image-711240.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq7LEbYb3OsxDtNLU59pbG9GiOReOuEil73sN1SK7hAHhD4YNld3kDyyJdsOaCGmalhZvSNb4lGbxir9RdNuwh70Mw1sI2HpblR_HyWh0hxvYRiboM7bu42BQjtwNx8n_Xer1XcUE2G2A/s320/image-711240.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6894917208949300866" /></a><br></font></div></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif">We could write more of the data and create more dashboards to provide more insights.</font></div><div><br></div></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-20204711237490074292020-11-13T19:11:00.001+05:302020-11-14T17:40:55.341+05:30Elasticsearch/Kubernetes/Logstash(ELK) - Part 3<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><b style=""><font face="trebuchet ms, sans-serif" size="4">Elastic Search Query & Practice</font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="font-family:"trebuchet ms",sans-serif">Everything in Elasticsearch is HTTP and the body uses the JSON format to query the Elasticsearch. This language to query elasticsearch is DSL which will match your search criteria and provide some kind of relevancy </span><span style="font-family:"trebuchet ms",sans-serif">score, more occurrences of the words in the document is considered to be more relevant(relevancy score).</span><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br><b>Syntax for DSL components</b><br><br><b>Query</b>: Query context which is used for full text searches and that's supposed to match documents that contain the search criteria as well to specify how well the document matched that particular search criteria by providing that relevancy score.<br><br><b>Filter</b>: <span style="color:rgb(33,37,41)">Filter context is mostly used for filtering structured data</span> <br><br>But both the query context and the filter context can be combined together to form one large query.<br><br><b><font size="4">Index Creation and Query using DSL</font></b></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font size="4" style="color:rgb(0,0,0)"><b><br></b></font><font color="#000000">Create these below 10 records and get hands on how to use query Elasticsearch. This is mainly for practice, create it one by one in the kibal console which you can practice for querying the searches.</font><br><br><font color="#741b47">PUT /courses/classroom/1<br>{<br> "name": "Accounting 101",<br> "room": "E3",<br> "professor": {<br> "name": "Thomas Baszo",<br> "department": "finance",<br> "facutly_type": "part-time",<br> "email": "<a href="mailto:baszot@onuni.com">baszot@onuni.com</a>"<br> },<br> "students_enrolled": 27,<br> "course_publish_date": "2015-01-19",<br> "course_description": "Act 101 is a course from the business school on the introduction to accounting that teaches students how to read and compose basic financial statements"<br>}<br><br>PUT /courses/classroom/2<br>{<br> "name": "Marketing 101",<br> "room": "E4",<br> "professor": {<br> "name": "William Smith",<br> "department": "finance",<br> "facutly_type": "part-time",<br> "email": "<a href="mailto:wills@onuni.com">wills@onuni.com</a>"<br> },<br> "students_enrolled": 18,<br> "course_publish_date": "2015-06-21",<br> "course_description": "Mkt 101 is a course from the business school on the introduction to marketing that teaches students the fundamentals of market analysis, customer retention and online advertisements"<br>}<br><br>PUT /courses/classroom/3<br>{<br> "name": "Anthropology 230",<br> "room": "G11",<br> "professor": {<br> "name": "Devin Cranford",<br> "department": "history",<br> "facutly_type": "full-time",<br> "email": "<a href="mailto:devinc@onuni.com">devinc@onuni.com</a>"<br> },<br> "students_enrolled": 22,<br> "course_publish_date": "2013-08-27",<br> "course_description": "Ant 230 is an intermediate course on human societies and cultures and their development. A focus on the Mayans civilization is rooted in this course"<br>}<br><br>PUT /courses/classroom/4<br>{<br> "name": "Computer Science 101",<br> "room": "C12",<br> "professor": {<br> "name": "Gregg Payne",<br> "department": "engineering",<br> "facutly_type": "full-time",<br> "email": "<a href="mailto:payneg@onuni.com">payneg@onuni.com</a>"<br> },<br> "students_enrolled": 33,<br> "course_publish_date": "2013-08-27",<br> "course_description": "CS 101 is a first year computer science introduction teaching fundamental data structures and alogirthms using python. "<br>}<br><br>PUT /courses/classroom/5<br>{<br> "name": "Theatre 410",<br> "room": "T18",<br> "professor": {<br> "name": "Sebastian Hern",<br> "department": "art",<br> "facutly_type": "part-time",<br> "email": ""<br> },<br> "students_enrolled": 47,<br> "course_publish_date": "2013-01-27",<br> "course_description": "Tht 410 is an advanced elective course disecting the various plays written by shakespere during the 16th century"<br>}<br><br>PUT /courses/classroom/6<br>{<br> "name": "Cost Accounting 400",<br> "room": "E7",<br> "professor": {<br> "name": "Bill Cage",<br> "department": "accounting",<br> "facutly_type": "full-time",<br> "email": "<a href="mailto:cageb@onuni.com">cageb@onuni.com</a>"<br> },<br> "students_enrolled": 31,<br> "course_publish_date": "2014-12-31",<br> "course_description": "Cst Act 400 is an advanced course from the business school taken by final year accounting majors that covers the subject of business incurred costs and how to record them in financial statements"<br>}<br><br>PUT /courses/classroom/7<br>{<br> "name": "Computer Internals 250",<br> "room": "C8",<br> "professor": {<br> "name": "Gregg Payne",<br> "department": "engineering",<br> "facutly_type": "part-time",<br> "email": "<a href="mailto:payneg@onuni.com">payneg@onuni.com</a>"<br> },<br> "students_enrolled": 33,<br> "course_publish_date": "2012-08-20",<br> "course_description": "cpt Int 250 gives students an integrated and rigorous picture of applied computer science, as it comes to play in the construction of a simple yet powerful computer system. "<br>}<br><br>PUT /courses/classroom/8<br>{<br> "name": "Accounting Info Systems 350",<br> "room": "E3",<br> "professor": {<br> "name": "Bill Cage",<br> "department": "accounting",<br> "facutly_type": "full-time",<br> "email": "<a href="mailto:cageb@onuni.com">cageb@onuni.com</a>"<br> },<br> "students_enrolled": 19,<br> "course_publish_date": "2014-05-15",<br> "course_description": "Act Sys 350 is an advanced course providing students a practical understanding of an accounting system in database technology. Students will use MS Access to build a transaction ledger system"<br>}<br><br>PUT /courses/classroom/9<br>{<br> "name": "Tax Accounting 200",<br> "room": "E7",<br> "professor": {<br> "name": "Thomas Baszo",<br> "department": "finance",<br> "facutly_type": "part-time",<br> "email": "<a href="mailto:baszot@onuni.com">baszot@onuni.com</a>"<br> },<br> "students_enrolled": 17,<br> "course_publish_date": "2016-06-15",<br> "course_description": "Tax Act 200 is an intermediate course covering various aspects of tax law"<br>}<br><br>PUT /courses/classroom/10<br>{<br> "name": "Capital Markets 350",<br> "room": "E3",<br> "professor": {<br> "name": "Thomas Baszo",<br> "department": "finance",<br> "facutly_type": "part-time",<br> "email": "<a href="mailto:baszot@onuni.com">baszot@onuni.com</a>"<br> },<br> "students_enrolled": 13,<br> "course_publish_date": "2016-01-11",<br> "course_description": "This is an advanced course teaching crucial topics related to raising capital and bonds, shares and other long-term equity and debt financial instrucments"<br>}</font><br><br><font color="#000000">The most basic kind of query in a classic search is the match or query. </font><font color="#000000">You can see the _score to find the relevance of the document.</font></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000">There are more examples in this section for practice purposes.<br></font><br><font color="#741b47">GET /courses/_search</font><br><font color="#741b47">{</font><br><font color="#741b47"> "query":{</font><br><font color="#741b47"> "match_all": {}</font><br><font color="#741b47"> }</font><br><font color="#741b47">}</font><br><br><font color="#741b47">GET /courses/_search</font><br><font color="#741b47">{</font><br><font color="#741b47"> "query":{</font><br><font color="#741b47"> "match": {"name":"computer"}</font><br><font color="#741b47"> }</font><br><font color="#741b47">}</font><br><br><font color="#741b47">PUT /courses/classroom/5</font><br><font color="#741b47">{</font><br><font color="#741b47"> "name": "Theatre 410",</font><br><font color="#741b47"> "room": "T18",</font><br><font color="#741b47"> "professor": {</font><br><font color="#741b47"> "name": "Sebastian Hern",</font><br><font color="#741b47"> "department": "art",</font><br><font color="#741b47"> "facutly_type": "part-time"</font><br><font color="#741b47"> },</font><br><font color="#741b47"> "students_enrolled": 47,</font><br><font color="#741b47"> "course_publish_date": "2013-01-27",</font><br><font color="#741b47"> "course_description": "Tht 410 is an advanced elective course </font>disecting<font color="#741b47"> the various plays written by </font>shakespere<font color="#741b47"> during the 16th century"</font><br><font color="#741b47">}</font><br><br><font color="#000000">displays all the records which exists in the field</font><br><br><font color="#741b47">GET /courses/_search</font><br><font color="#741b47">{</font><br><font color="#741b47"> "query":{</font><br><font color="#741b47"> "exists": {"field":"professor.email"}</font><br><font color="#741b47"> }</font><br><font color="#741b47">}</font><br><br><font color="#000000">Match multiple criteria</font><br><br><font color="#741b47">GET /courses/_search<br>{<br> "query":{<br> "bool": {<br> "must": [<br> {"match": {"name":"computer"}},<br> {"match": {"room": "c8"}}<br> ]<br> }<br> }<br>}</font><br><br><font color="#000000">Multi match criteria</font><br><br><font color="#741b47">GET /courses/_search<br>{<br> "query":{<br> "multi_match":{<br> "fields": ["name","<a href="http://professor.name">professor.name</a>"],<br> "query": "accounting"<br> }<br> }<br>}</font><br><br><font color="#000000">Searches for the string with exact words</font><br><br><font color="#741b47">GET /courses/_search<br>{<br> "query":{<br> "match_phrase":{<br> "course_description": "financial statements"<br> }<br> }<br>}</font><br><br><font color="#000000">Searches for the words with partial contexts also.</font><br><br><font color="#741b47">GET /courses/_search<br>{<br> "query":{<br> "match_phrase_prefix":{<br> "course_description": "financial statements"<br> }<br> }<br>}</font><br><br><font color="#000000">Search in range</font><br><br><font color="#741b47">GET courses/_search<br>{<br> "query": {<br> "range": {<br> "students_enrolled": {<br> "gte": 10,<br> "lte": 15<br> }<br> }<br> }<br>}<br></font><br><font color="#000000">Search for date</font><br><br><font color="#741b47">GET courses/_search<br>{<br> "query": {<br> "range": {<br> "course_publish_date": {<br> "gte": 2013<br> }<br> }<br> }<br>}</font><br><br><font color="#000000">Combine two or more queries.</font><br><br><font color="#741b47">GET courses/_search<br>{<br> "query": {<br> "bool": {<br> "must": [<br> {"match": {<br> "name": "accounting"<br> }}<br> ]<br> , "must_not": [<br> {"match": {<br> "room": "e7"<br> }}<br> ]<br> , "should": [<br> {"range": {<br> "students_enrolled": {<br> "gte": 10,<br> "lte": 20<br> }<br> }}<br> ]<br> }<br> }<br>}</font><br><br><font color="#000000"><b>Filters</b></font><br><br><font color="#741b47">GET courses/_search<br>{<br> "query": {<br> "bool": {<br> "filter": {<br> "bool": {<br> "must":[<br> {<br> "range":{<br> "students_enrolled":{<br> "gte": 30<br> }<br> }<br> }<br> ]<br> }<br> }<br> }<br> }<br>}</font><br><br>You can create a new set of index, to practice aggregations and filter.</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000"><b>Bulk Indexing</b></font><br><br><font color="#990000">POST /vehicles/cars/_bulk<br>{ "index": {}}<br>{ "price" : 10000, "color" : "white", "make" : "honda", "sold" : "2016-10-28", "condition": "okay"}<br>{ "index": {}}<br>{ "price" : 20000, "color" : "white", "make" : "honda", "sold" : "2016-11-05", "condition": "new" }<br>{ "index": {}}<br>{ "price" : 30000, "color" : "green", "make" : "ford", "sold" : "2016-05-18", "condition": "new" }<br>{ "index": {}}<br>{ "price" : 15000, "color" : "blue", "make" : "toyota", "sold" : "2016-07-02", "condition": "good" }<br>{ "index": {}}<br>{ "price" : 12000, "color" : "green", "make" : "toyota", "sold" : "2016-08-19" , "condition": "good"}<br>{ "index": {}}<br>{ "price" : 18000, "color" : "red", "make" : "dodge", "sold" : "2016-11-05", "condition": "good" }<br>{ "index": {}}<br>{ "price" : 80000, "color" : "red", "make" : "bmw", "sold" : "2016-01-01", "condition": "new" }<br>{ "index": {}}<br>{ "price" : 25000, "color" : "blue", "make" : "ford", "sold" : "2016-08-22", "condition": "new" }<br>{ "index": {}}<br>{ "price" : 10000, "color" : "gray", "make" : "dodge", "sold" : "2016-02-12", "condition": "okay" }<br>{ "index": {}}<br>{ "price" : 19000, "color" : "red", "make" : "dodge", "sold" : "2016-02-12", "condition": "good" }<br>{ "index": {}}<br>{ "price" : 20000, "color" : "red", "make" : "chevrolet", "sold" : "2016-08-15", "condition": "good" }<br>{ "index": {}}<br>{ "price" : 13000, "color" : "gray", "make" : "chevrolet", "sold" : "2016-11-20", "condition": "okay" }<br>{ "index": {}}<br>{ "price" : 12500, "color" : "gray", "make" : "dodge", "sold" : "2016-03-09", "condition": "okay" }<br>{ "index": {}}<br>{ "price" : 35000, "color" : "red", "make" : "dodge", "sold" : "2016-04-10", "condition": "new" }<br>{ "index": {}}<br>{ "price" : 28000, "color" : "blue", "make" : "chevrolet", "sold" : "2016-08-15", "condition": "new" }<br>{ "index": {}}<br>{ "price" : 30000, "color" : "gray", "make" : "bmw", "sold" : "2016-11-20", "condition": "good" }<br></font><br><font color="#000000" size="4"><b>Aggregation</b></font><br><font color="#000000">till now we were only searching, now we would do data analytics by insights into data.</font><br><br><font color="#000000">Search all data with price in descending order.</font><br><br><font color="#741b47">GET /vehicles/cars/_search<br>{<br> "from": 0,<br> "size": 5<br><br> , "query": {<br> "match_all": {}<br> }<br> , "sort": [<br> {<br> "price": {<br> "order": "desc"<br> }<br> }<br> ]<br>}</font><br><br><font color="#000000">Creating the Aggregation which searches </font>on all<font color="#000000"> the cars which are in color 'red' with price details.</font><br><br><font color="#741b47">GET /vehicles/cars/_search<br>{<br> "size": 1,<br> "query": {<br> "match": {<br> "color": "red"<br> }<br> },<br><br> "aggs": {<br> "popular_cars": {<br> "terms": {<br> "field": "make.keyword",<br> "size": 10<br> }<br> , "aggs": {<br> "avg_price": {<br> "avg": {<br> "field": "price"<br> }<br> },<br> "max_price": {<br> "max": {<br> "field": "price"<br> }<br> },<br> "min_price": {<br> "min": {<br> "field": "price"<br> }<br> }<br> }<br> }<br> }<br>}<br></font><br><font color="#000000">Creating the buckets for the sold date.</font><br><br><font color="#741b47">GET /vehicles/cars/_search<br>{<br> "aggs": {<br> "popular_cars": {<br> "terms": {<br> "field": "make.keyword",<br> "size": 10<br> }<br> , "aggs": {<br> "sold_date_range": {<br> "range": {<br> "field": "sold",<br> "ranges": [<br> {<br> "from": "2016-01-01",<br> "to": "2016-05-18"<br> },<br> {<br> "from": "2016-05-18",<br> "to": "2017-01-01"<br> }<br> ]<br> },<br> "aggs": {<br> "avg_price": {<br> "avg": {<br> "field": "price"<br> }<br> }<br> }<br> }<br> }<br> }<br> }<br>}</font><br><br><font color="#000000">Find the conditions of the cars, their max, min price as an aggregate scoping.</font><br><br><font color="#741b47">GET /vehicles/cars/_search<br>{<br> "aggs": {<br> "car_condition": {<br> "terms": {<br> "field": "condition.keyword",<br> "size": 10<br> }<br> , "aggs": {<br> "avg_price": {<br> "avg": {<br> "field": "price"<br> }<br> },<br> "make":{<br> "terms": {<br> "field": "make.keyword",<br> "size": 10<br> },<br> "aggs":{<br> "min_price": { "min": { "field": "price" }},<br> "max_price": { "max": { "field": "price" }}<br> }<br> }<br> }<br> }<br> }<br>}</font></font><br></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#741b47"><br></font></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#741b47"><br></font></font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-70976621457023117982020-11-13T07:36:00.001+05:302020-11-14T17:41:00.263+05:30Elasticsearch/Kubernetes/Logstash(ELK) - Part 2<div dir="ltr"><div class="gmail_default" style="color: black;"><font face="trebuchet ms, sans-serif"><font size="4"><b>Index Creations</b></font><br /><br />What happens when we send requests to index a document or query the document in a classic search ?<br /><br />As you know elastic search is a distributed technology, which means it's faster to solve the given problem. So if we have a really large search problem and we break it up into really small ones and we hand them out to each one of those computers and all those computers can sort of work in parallel to solve the problem in a much shorter period of time as opposed to just one computer.<br /><br />index search is actually very scalable and the logical representation for any human readable data, under the hood it's actually called as <b>shards</b>. So a particular index like this could be split up into multiple shards.<br /><br />example,<br />document from 1 to 50 -> shard_0<br />document from 51 to 100 -> Shard_1<br /><br />Shards, since we already know that these are distributed(one or more computers) so node one is one computer and Node 2 is another computer and both of them have a classic search running and they can communicate with one another, when on the same network(clustering). so smaller units of storage is called "<b>shards</b>"<br /><br />shards are placed in either of the containers(computers) so that when actual search requests come up, any of the nodes in the cluster would be able to respond. For backup of these, you call as <b>replica shards</b> if required to be configured. so when creating an index, it will be an exact copy of the actual shard, this is mainly used in the production.<br /><br />How does search take place ?<br /><br />what node is going to do is take the ID of this document that we are trying to index and it's going to run it through something called the hashing function.<br /><br /><b>Shards</b><br />So a shard is basically a container of inverted indices also called segments. A segment belongs inside of charts, and a chart can have multiple segments and each one of these segments or inverted indices.<br /><br /><b>inverted index:</b><br />long list of words in alphabetical order and next to each word we need to put the document that word occurred the actual location for that, called token and the process of doing it is called tokenization.<br /><br />And as you're aware and an elastic search index is made up of multiple shards, and that is how an index can span multiple nodes because the unit of separation is the Shard.<br /><br />The process of taking raw tax and converting it and turning it into an inverted index is called <b>analysis</b>, so that data is searchable which makes it faster. Once it forms this inverted index then it gets into the memory buffer.<br /><br />so this one document we sent it to a logic search for indexing. It went through the analysis step and inverted indexes created this document are sent to the buffer and get populated with that document, in the same way, another document to classic search goes through the analysis process and inverted indexes formed and it sends it into the buffer. Once this buffer gets filled up then this buffer gets committed to a <b>segment</b>. which are not called as <b>immutable index</b><br /><br />And this is the basis of setting your data so that it's searchable now. Once the segments are done this chart is now searchable. It's searchable and you can rest assured that the data in here is not going to be changed; it's been processed through analysis; inverted indices were formed and then they were committed to the segment. So this is permanent data that you can know you can search and that's all. Each one of those shards are formed.<br /><br />So the magical process that happens here is this analysis step that's what turns a document like this into an inverted index.<br /><br /><font size="4"><b>Analysis Process</b></font><br /><br />When we send a document into elasticsearch goes through a process of running this analysis step. The objective of this step is to convert or transform the document into an inverted index and store it into a shard, so this inverted index gets put into a shard segment. So this process of analysis is the key in indexing documents and not only during indexing but also during Query time when we retrieve or read the documents.<br /><br />Example:<br />we will put these two sentences into analysis process<br /><br />"sentence 1"<br />"sentence 2"<br /><br />We were indexing these two sentences. We'd need to get rid of unnecessary information so that we can get to the most important pieces of both<br />of these documents and only those pieces would be indexed so if we were to convert these documents into an inverted index. This is called <b>tokenization</b>.<br /><br />It goes through this process called an analyzer, who does all of the analysis and has two parts:<br /><br />- The first part is tokenization So it has a tokenizer.<br />- The second step is filtering.<br /><br />these steps are being done at the filtering,<br />- Remove stop words (whitespaces)<br />- Lowercasing<br />- Stemming (eg running, run, swimming, swim)<br />- Synonyms (eg thin, lean, skinny)<br /><br />And when the text goes into this analyzer let's say we are indexing right we are indexing the document when text goes into this analyzer it gets first tokenized and now filtering takes place the tokens that come out are the ones that make it into the inverted index. So this is the indexing step.</font><br /></div><div class="gmail_default" style="color: black;"><font face="trebuchet ms, sans-serif"><br /></font></div><div class="gmail_default"><font face="trebuchet ms, sans-serif"><div class="gmail_default"><span style="font-size: medium;">Define Custom Index Structure</span></div><div class="gmail_default"><br /></div><div class="gmail_default">we could define an index "hr" we deal with this particular structure whichis the logical representation</div><div class="gmail_default">of the actual data the actual data resides on the disk called shards. We're usually concerned with the logical</div><div class="gmail_default">representation of the index and how to load the data into index.</div><div class="gmail_default"><br /></div><div class="gmail_default">Now let's get into the details of how the index structure can be defined if you had to create your index</div><div class="gmail_default">manually and define the different fields in the properties and so on. So far elastic search created an index</div><div class="gmail_default">structure for us dynamically on the fly. Now, lets see how to manually create it.</div><div class="gmail_default"><br /></div><div class="gmail_default">PUT /customer</div><div class="gmail_default">{</div><div class="gmail_default"> "settings" : {</div><div class="gmail_default"> "number_of_replicas" : 2,</div><div class="gmail_default"> "number_of_shards": 1</div><div class="gmail_default"> },</div><div class="gmail_default"> "mappings" : {</div><div class="gmail_default"> "online" : {</div><div class="gmail_default"> "properties" : {</div><div class="gmail_default"> "gender" : {</div><div class="gmail_default"> "type" : "text",</div><div class="gmail_default"> "analyzer": "standard"</div><div class="gmail_default"> },</div><div class="gmail_default"> "age": {</div><div class="gmail_default"> "type": "integer"</div><div class="gmail_default"> },</div><div class="gmail_default"> "total_spent": {</div><div class="gmail_default"> "type": "float"</div><div class="gmail_default"> }</div><div class="gmail_default"> }</div><div class="gmail_default"> }</div><div class="gmail_default"> }</div><div class="gmail_default">}</div><div class="gmail_default"><br /></div><div class="gmail_default">Lets create data to our index, as what we defined earlier.</div><div class="gmail_default"><br /></div><div class="gmail_default">PUT /customer/online/2343</div><div class="gmail_default">{</div><div class="gmail_default"> "gender": "male",</div><div class="gmail_default"> "age": 22,</div><div class="gmail_default"> "total_spent": 50000,</div><div class="gmail_default"> "location": "Kashmir"</div><div class="gmail_default">}</div><div class="gmail_default"><br /></div><div class="gmail_default">notice that when we use GET /customer all the data, even though what ever we have mentioned while defining</div><div class="gmail_default">index, elastic search was added itself dynamically with "location".</div><div class="gmail_default"><br /></div><div class="gmail_default">we could restrict Elasticsearch, to set value of dynamic</div><div class="gmail_default">- false: indexing field will be ignored.</div><div class="gmail_default">- strict: indexing field will throw error.</div><div class="gmail_default"><br /></div><div class="gmail_default">PUT /customer/_mapping/online</div><div class="gmail_default">{</div><div class="gmail_default"> "dynamic": "strict"</div><div class="gmail_default">}</div><div class="gmail_default"><br /></div><div class="gmail_default"><span style="font-size: medium;">Analyzers</span></div><div class="gmail_default"><br /></div><div class="gmail_default">Elastic search has wide range of built-in analyzers, which can be used in any index without further configurations.</div><div class="gmail_default">https://www.elastic.co/guide/en/elasticsearch/reference/6.8/analysis-analyzers.html</div><div style="color: black;"><br /></div></font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-32207016286521213182020-11-12T18:34:00.001+05:302020-11-14T17:41:10.001+05:30Elasticsearch/Kubernetes/Logstash(ELK) - Part 1<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">In this article we will learn about ELK. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Installing Elasticsearch & Kibana:<br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">As part of the prerequisite, ensure you have installed Java.<br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Download Elastic and Kibana from below release page</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><a rel="noopener noreferrer" href="https://www.elastic.co/downloads/past-releases" style="box-sizing:border-box;color:rgb(0,119,145);text-decoration-line:none">https://www.elastic.co/downloads/past-releases</a> </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><a rel="noopener noreferrer" href="https://www.elastic.co/downloads/past-releases" style="box-sizing:border-box;color:rgb(0,119,145);text-decoration-line:none">https://www.elastic.co/downloads/past-releases</a><span style="color:rgb(41,48,59)">.</span></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">unzip the downloaded file for both.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">cd kibana*</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">vim config/kibana.yml</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">search for <b>elasticsearch.url </b> in the config file and uncomment the line. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">it would be default pointed to elasticsearch at localhost:9200</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Running Elasticsearch and Kibana</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">First always start the elasticsearch, <b>bin/elasticsearch.sh</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">second, start kibana, <b>bin/kibana.sh</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Open your browser and point to urls,</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000">Elasticsearch: </font><font color="#0000ff"><a href="http://localhost:9200">http://localhost:9200</a> </font><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000">Kibana: </font><font color="#0000ff"><a href="http://localhost:5601">http://localhost:5601</a></font></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000">As part of practice working ELK, we can populate data into a classic search, retrieve data, and delete data.</font><br><font color="#000000">In the elastic search rolled data is stored into something called an index.</font><br><br><font color="#000000">We would be taking an example of HR index and will create an index called hr and will store employee type and each employee would be created with an id.</font><br><font color="#000000">e.g</font><br><font color="#000000"><index>/<type>/<name></font><br><font color="#000000">/hr/employee/xyz</font><br><br><font color="#0000ff">PUT /hr/employees/sunil</font><br><font color="#0000ff">{<br> "Name": "Sunil",<br> "EmpID": "123"<br>}</font><br><br><font color="#000000">Returns, the success code of the API call.</font><br><font color="#0000ff">HEAD /hr/employees/sunil</font><br><br><font color="#000000">Retrieve, data</font><br><font color="#0000ff">GET /hr/employees/sunil</font><br><br><font color="#000000">Update data,</font><br><font color="#0000ff">POST /hr/employees/sunil/_update</font><br><font color="#0000ff">{<br> "doc":{<br> "Location": "Bengaluru"<br> }<br>}</font><br><br><font color="#000000">whenever data is being written it would not just change the attributes, instead the document itself.</font><br><br><font color="#000000">Delete data,</font><br><font color="#0000ff">DELETE /hr/employees/sunil</font><br><font color="#000000">The deletion only did on the attribute on the call, however the index still remains.</font><br><br><font color="#0000ff">DELETE /hr</font><br><br><font color="#000000" size="4"><b>Index Components</b></font></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000" size="4"><br></font><font color="#0000ff">GET /business</font><br><font color="#000000">You </font>won't<font color="#000000"> have any index hence it returns error, we will try to create a new index.</font><br><br><font color="#0000ff">PUT /business/building/200</font><br><font color="#0000ff">{<br> "address": "498 Dave Street In",<br> "floors": 3,<br> "offices": 5,<br> "loc": {<br> "latitude": 23.2332,<br> "longitute": 34.23233<br> }<br>}</font><br><br><font color="#0000ff">GET /business</font><br><font color="#000000">You would get the below output which has main </font>componets<font color="#000000"> as</font><br><font color="#000000">aliases, mappings, settings.</font><br><br><font color="#000000">So when we try to add more records into the search with different fields, elastics would map itself to the mapping section.</font><br><font color="#000000">elastic search dynamic</font><br><br><font color="#000000">{</font><br><font color="#000000"> "business": {</font><br><font color="#000000"> "aliases": {},</font><br><font color="#000000"> "mappings": {},</font><br><font color="#000000"> "settings": {}</font><br><font color="#000000"> }</font><br><font color="#000000">}</font><br><br><font color="#0000ff">PUT /business/building/201</font><br><font color="#0000ff">{<br> "address": "498 Dave Street In",<br> "floors": 3,<br> "offices": 5,<br> "price": 5000000,<br> "loc": {<br> "latitude": 23.2332,<br> "longitute": 34.23233<br> }<br>}</font><br><br><font color="#000000">Note: we could only have 1 type in the index.</font><br><font color="#000000">e.g PUT /business/employees/232, this would give an error as /business is already associated with "buildings"</font><br><font color="#000000">so you can </font>crate<font color="#000000"> in this way, with new Index</font><br><br><font color="#0000ff">PUT /employees/_doc/200<br>{<br> "Name": "Sunil",<br> "title": "Senior Engineer",<br> "joining_data": "Jan 01 2020"<br>}<br></font><br><font color="#0000ff">PUT /employees/_doc/201<br>{<br> "Name": "Ram",<br> "title": "Senior Tech Engineer",<br> "joining_data": "Jul 01 2000"<br>}</font><br><br><font color="#0000ff">PUT /contracts/_doc/220<br>{<br> "Name": "System Admins",<br> "start_date": "Jan 10 2015",<br> "employees": [200, 201]<br>}<br></font><br><br><font color="#000000"><b>Query data</b></font><br><br><font color="#0000ff">GET business/building/_search<br> or<br>GET business/_search</font><br><br><font color="#000000">Search and get only the required record</font><br><font color="#0000ff">GET business/_search<br>{<br> "query": {<br> "term": {<br> "address": "498"<br> }<br> }<br>}</font><br><br><font color="#000000">Actual request for elastic search which goes from kibana console would be like below,</font><br><br><font color="#0000ff">curl -X GET "<a href="http://localhost:9200/business/_search?pretty">http://localhost:9200/business/_search?pretty</a>" -H 'Content-Type: application/json' -d'<br>{<br> "query": {<br> "term": {<br> "address": "498"<br> }<br> }<br>}'</font><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">We will further check on the next parts on text analysis for indexing and searching etc.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-60935016183815019722020-11-08T12:40:00.001+05:302020-11-08T16:44:35.471+05:30Prometheus Monitoring - Part 5<div dir="ltr"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"></span><font face="trebuchet ms, sans-serif"><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">My previous articles mainly discussed regarding the alerts, alerts manager, and notifications. </span></font></div><div><a href="https://sunlnx.blogspot.com/2020/11/prometheus-monitoring-part-4.html">https://sunlnx.blogspot.com/2020/11/prometheus-monitoring-part-4.html</a><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></span></font></div><div><br></div><div><font face="trebuchet ms, sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">In this article, we will include Prometheus as a datasource from Grafana for data visualisations. </span></font></div></font><div><ul><li><font face="trebuchet ms, sans-serif"><b style="color:rgb(0,0,0)"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"></span>Install & Configure <span class="gmail_default">Grafana</span></b></font></li><ul><li><span style="color:rgb(0,0,0);font-family:"trebuchet ms",sans-serif">Install nginx</span></li><li><span style="color:rgb(0,0,0);font-family:"trebuchet ms",sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Install & Configure Nginx </span></span></li><li><span style="color:rgb(0,0,0);font-family:"trebuchet ms",sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Configure Nginx reverse proxy</span></span></li><li><span style="color:rgb(0,0,0);font-family:"trebuchet ms",sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Configure SSL</span></span></li><li><span style="color:rgb(0,0,0);font-family:"trebuchet ms",sans-serif"><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">Register to DNS</span></span></li></ul></ul></div><div><ul><li><font face="trebuchet ms, sans-serif"><b><span style="color:rgb(0,0,0)"><font color="#000000" style=""></font></span><span style="color:rgb(34,34,34)">Setup Prometheus DataSource</span></b></font></li><li><font face="trebuchet ms, sans-serif"><b>Setup Prometheus Dashboards</b></font></li><li><font face="trebuchet ms, sans-serif"><b>Create dashboards for node exporters </b></font></li></ul><div><font face="trebuchet ms, sans-serif"><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><span style="font-size:large"></span><span class="gmail_default" style="font-size:large"></span><span style="font-size:large;color:rgb(34,34,34)">Install & Configure </span><span class="gmail_default" style="font-size:large;color:rgb(34,34,34)">Grafana</span><span style="font-size:large"></span></div></font></div></div><font face="trebuchet ms, sans-serif"><div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><span style="color:rgb(34,34,34)">I am installing Grafana using Ubuntu 20.04, with a root account.</span><br></div></div></font><div><font face="trebuchet ms, sans-serif"><br><font color="#0000ff">sudo apt update<br>sudo apt-get install -y adduser libfontconfig1<br>wget <a href="https://dl.grafana.com/oss/release/grafana_7.2.0_amd64.deb">https://dl.grafana.com/oss/release/grafana_7.2.0_amd64.deb</a><br>sudo dpkg -i grafana_7.2.0_amd64.deb<br>sudo service grafana-server start<br>sudo systemctl enable grafana-server.service</font><br><br></font></div><div><font face="trebuchet ms, sans-serif">Your Grafana server will be hosted at<span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"> </span>http://[your Grafana server ip]:3000<br>The default Grafana login is<br>Username :admin<br>Password :admin<br><br>If you need to have an SSL, installing an nginx proxy would be fine and then configure reverse proxy to redirect accordingly.<br><br><font color="#0000ff">sudo apt install nginx -y<br>sudo vim /etc/nginx/sites-enabled/prometheus</font><br><br><font color="#741b47">server {<br> listen 80;<br> listen [::]:80;<br> server_name prometheus.YOUR-DOMAIN-NAME;<br><br> location / {<br>proxy_pass <a href="http://localhost:3000/">http://localhost:3000/</a>;<br> }<br>}</font><br><br>Save and test the new configuration has no errors<br>nginx -t<br><br><a href="http://YOUR-DOMAIN-NAME">http://YOUR-DOMAIN-NAME</a><br>Visiting your ip address directly will still show the default Nginx welcome page. you can remove<span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">(</span><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"> </span><font color="#0000ff">rm /etc/nginx/sites-enabled/default<span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"> )</span></font><br><br>restart nginx,<br><font color="#0000ff">sudo service nginx restart<br>sudo service nginx status</font><br><br>Add SSL certificates to the grafana dashboards.<br><br><font color="#0000ff">sudo snap install --classic certbot<br>sudo certbot --nginx<br></font><br></font></div><div><font face="trebuchet ms, sans-serif">Once those certs are installed, you can use <a href="https://grafana.domainname.com">https://grafana.domainname.com</a> to login from the browser.</font></div><div><span style="color:rgb(0,0,0);font-family:"trebuchet ms",sans-serif"></span><font face="trebuchet ms, sans-serif"><br></font></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWX8XsZcIOV0r3wc1ZTfzRdymm5oSIX0Oc-oa0uegyMDUdN1ufXNSK4ezevjjqn6WmtTCfO1nUlhzM8cm-TB_QHqGMOK0ttVUjHm2PO6Lo_s9r2h-LI5dAOS7JPZHfhLA9a0LTiylck6o/s1600/image-723076.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWX8XsZcIOV0r3wc1ZTfzRdymm5oSIX0Oc-oa0uegyMDUdN1ufXNSK4ezevjjqn6WmtTCfO1nUlhzM8cm-TB_QHqGMOK0ttVUjHm2PO6Lo_s9r2h-LI5dAOS7JPZHfhLA9a0LTiylck6o/s320/image-723076.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646942022828530" /></a><br></div><div><br></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" size="4"><b><font color="#000000" style=""></font>Setup Prometheus DataSource</b></font><br><br><font face="trebuchet ms, sans-serif">Once you logged into the grafana, go to "Configurations" → Click on "DataSources" → Select "Prometheus" .</font><br><font face="trebuchet ms, sans-serif">Configurations required over here have to be filled up. </font><br></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style=""><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHwTUd-vEXlX4OzcN0C6Mnt5HQp0UFkkfmwwaGb_eB-NnO9AmzxHW4TkdOQvtF_wMdzfCZa2J6gBZcOpUbWb-tzUwsbQcqPb6LAbFO3_Q46UDAfnvOptZvvsDEl-iyBNwtk1GDeKzfvEo/s1600/image-725137.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHwTUd-vEXlX4OzcN0C6Mnt5HQp0UFkkfmwwaGb_eB-NnO9AmzxHW4TkdOQvtF_wMdzfCZa2J6gBZcOpUbWb-tzUwsbQcqPb6LAbFO3_Q46UDAfnvOptZvvsDEl-iyBNwtk1GDeKzfvEo/s320/image-725137.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646947270535186" /></a><br></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Vy3YEg9c79QrXL3AB5aJFy9BGPKuFw5JUhRc-Yy98_rW6Mo8ypV4XlDTGUCkZ4bmpbXQfu2PHt_YVRI54g_Q6Cmjlbqp-1yd1mb4m1iK2V0eh5_janKPNSfrAdOa91TPpqT8bIK0Paw/s1600/image-726488.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Vy3YEg9c79QrXL3AB5aJFy9BGPKuFw5JUhRc-Yy98_rW6Mo8ypV4XlDTGUCkZ4bmpbXQfu2PHt_YVRI54g_Q6Cmjlbqp-1yd1mb4m1iK2V0eh5_janKPNSfrAdOa91TPpqT8bIK0Paw/s320/image-726488.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646951132247618" /></a><br></div></div></div><br></div><div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><span style="color:rgb(34,34,34)">Go to the explore tabs in which Prometheus is already selected, and run the query "go_threads". </span></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><span style="color:rgb(34,34,34)"><br></span></font></div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwdU3vUAcidmFT5tA5LRweSTeTJkp1MtXytXTz1DWOahRQqFFFbrXI8K_fbBtnYzPkZbd9enf9hi5m2ouovIZys3OiYcRyfw8nykJGAYXzPFQC0oDdD0oRkd-YPFSIdjR9S7Jz9jhgSRU/s1600/image-727913.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwdU3vUAcidmFT5tA5LRweSTeTJkp1MtXytXTz1DWOahRQqFFFbrXI8K_fbBtnYzPkZbd9enf9hi5m2ouovIZys3OiYcRyfw8nykJGAYXzPFQC0oDdD0oRkd-YPFSIdjR9S7Jz9jhgSRU/s320/image-727913.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646962375987522" /></a><br></div><div><br></div><div><font face="trebuchet ms, sans-serif"><font size="4"><b><span class="gmail_default" style="color:rgb(0,0,0)"></span>Setup Prometheus Dashboards</b></font><br><br>Go to Prometheus Configurations → Datasources → Click on the datasources which you have created → select Dashboards → Prometheus 2.0 Stats → Click on Import.</font></div><div><font face="trebuchet ms, sans-serif"><br></font></div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWQQNkkzMhF6W6oJNHFOZmlZReHzF-ZlBVr5pM5nUrw-Z-Nyd61rMs406TWff9ZD3_IX_3yG7mi-dEcFT0kBQkmB2MeD0Z8pa2KVjPBrmApuIYl7h3yWH-2-JsgQn8pGAncvqe_s2KYMY/s1600/image-729583.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWQQNkkzMhF6W6oJNHFOZmlZReHzF-ZlBVr5pM5nUrw-Z-Nyd61rMs406TWff9ZD3_IX_3yG7mi-dEcFT0kBQkmB2MeD0Z8pa2KVjPBrmApuIYl7h3yWH-2-JsgQn8pGAncvqe_s2KYMY/s320/image-729583.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646965379554194" /></a><br></div></div><font face="trebuchet ms, sans-serif"><div><font face="trebuchet ms, sans-serif"><br></font></div><b><span class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"></span><font size="4"><span class="gmail_default" style="color:rgb(0,0,0)"></span>Create dashboards for node exporters</font></b><br><br>Configurations section choose → Plugins → Click on "Find more plugins on Grafana.com" → Select "Dashboards" → Select [ English version ] → copy ID : 11074<br><br>Grafana Web page, select "Manage" from Dashboards → Select "Import" → Paste the ID : "11074" </font><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2MdCdDHNkI74qs-HwllMvmvhlU9P21C5JGlHn07DJ_hyphenhyphenDsYUt2TJT7xaf7RVVwznKMXNYkxkcNzYuDLcsfPaMwvTkrLxLvQ4dAZLy0mdLoUsPFFDKkp1yAkt4kAprb9ykPXIZRE1USuk/s1600/image-730937.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2MdCdDHNkI74qs-HwllMvmvhlU9P21C5JGlHn07DJ_hyphenhyphenDsYUt2TJT7xaf7RVVwznKMXNYkxkcNzYuDLcsfPaMwvTkrLxLvQ4dAZLy0mdLoUsPFFDKkp1yAkt4kAprb9ykPXIZRE1USuk/s320/image-730937.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646975044878402" /></a><br></div></div><div><br></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiORkUvcSm7C9-tzsLNSTq5Qbq-ZcFYFoLRbAfqzeqcB9SwAxwcz8UpSx_b8zQcsg0W19EUtPub6jN0bdsgpYFD1GEk4yXaP_tCWI167od3tIs05DgAZiHwtaeMxft7303yrP-7-ZIEMh0/s1600/image-732370.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiORkUvcSm7C9-tzsLNSTq5Qbq-ZcFYFoLRbAfqzeqcB9SwAxwcz8UpSx_b8zQcsg0W19EUtPub6jN0bdsgpYFD1GEk4yXaP_tCWI167od3tIs05DgAZiHwtaeMxft7303yrP-7-ZIEMh0/s320/image-732370.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892646978385456914" /></a><br></div></div><div><br></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-90741329776257281532020-11-08T09:08:00.001+05:302020-11-08T13:41:30.631+05:30Prometheus Monitoring - Part 4<div dir="ltr"><div class="gmail_default"> <font face="trebuchet ms, sans-serif">In previous post we discussed about PromQL and Alerts<br /><a href="https://sunlnx.blogspot.com/2020/11/promotheus-monitoring-part-3.html">https://sunlnx.blogspot.com/2020/11/promotheus-monitoring-part-3.html</a></font><p class="gmail-western" lang="en" style="color: black; line-height: 115%; margin-bottom: 0.35cm;"><font face="trebuchet ms, sans-serif">In this article, we will discuss these</font></p><font face="trebuchet ms, sans-serif"><ul><li><font face="trebuchet ms, sans-serif">Prometheus Alert Manager</font></li><li><font face="trebuchet ms, sans-serif">Configure SMTP local on Prometheus Server</font></li><li><font face="trebuchet ms, sans-serif">Test Alerts </font></li></ul></font><p class="gmail-western" lang="en" style="color: black; line-height: 115%; margin-bottom: 0.35cm;"> <font face="trebuchet ms, sans-serif" size="4"><b>Prometheus Alert Manager</b></font></p> <p class="gmail-western" lang="en" style="color: black; line-height: 115%; margin-bottom: 0.35cm;"><font face="trebuchet ms, sans-serif"><span style="color: #333333;">The AlertManager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.</span><br /></font></p> <p class="gmail-western" lang="en" style="color: black; line-height: 115%; margin-bottom: 0.15cm;"> <font face="trebuchet ms, sans-serif"><font><font><b>Install the Prometheus Alert Manager</b></font> </font> </font></p> <font face="trebuchet ms, sans-serif"><font color="#0000ff">sudo apt install prometheus-alertmanager<br />sudo service prometheus-alertmanager status<br />ps -u prometheus</font><br /></font><p class="gmail-western" style="color: black; line-height: 115%; margin-bottom: 0.15cm;"> <font color="#37474f"><font face="trebuchet ms, sans-serif" size="3">Note that the service is running on port 9093</font></font></p> <p class="gmail-western" style="color: black; margin-bottom: 0.21cm;"> <font face="trebuchet ms, sans-serif" size="3">Visit http://[your domain name or ip]:9093/</font></p> <div style="color: black;"><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_UcKLs7w55h9nGLVsZTkpsoBCQQCO-ku7-eg90-8qINgoVq5tnA_E5sWczZwC54QgHvqaUfRknVpTLO9ZNYo-ege48rt_QlexCh5IoGgDmYViS9Yzz5IggkOQrtG3TaHg1BAdQ-NACZk/s1600/image-719169.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6892592382278879218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_UcKLs7w55h9nGLVsZTkpsoBCQQCO-ku7-eg90-8qINgoVq5tnA_E5sWczZwC54QgHvqaUfRknVpTLO9ZNYo-ege48rt_QlexCh5IoGgDmYViS9Yzz5IggkOQrtG3TaHg1BAdQ-NACZk/s320/image-719169.png" /></a><br /></font></div><p class="gmail-western" style="color: black; margin-bottom: 0.21cm;"><span face=""trebuchet ms",sans-serif" style="color: #222222;">Block Port 9093 for external requests</span><br /></p><font color="#0000ff" face="trebuchet ms, sans-serif">iptables -A INPUT -p tcp -s localhost --dport 9093 -j ACCEPT<br />iptables -A INPUT -p tcp --dport 9093 -j DROP<br />iptables -L<br /><br />iptables-save > /etc/iptables/rules.v4<br />iptables-save > /etc/iptables/rules.v6</font><pre class="gmail-western" style="color: black; line-height: 115%; margin-bottom: 0.15cm;"></pre><font face="trebuchet ms, sans-serif"><font color="#0000ff">tail -4 prometheus.yml</font><br /><font color="#741b47">- job_name: alertmanager<br /> static_configs:<br /> - targets: ['localhost:9093']</font></font><p class="gmail-western" style="color: black; line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif"><span style="color: #37474f; font-size: 9pt;">verify the config and restart the service. Once they are successful, it must </span>display the target<span style="color: #37474f; font-size: 9pt;"> in the </span>Prometheus<span style="color: #37474f; font-size: 9pt;"> UI.</span></font></p><p class="gmail-western" style="color: black; line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif"><span style="color: #37474f; font-size: 9pt;"><br /></span></font></p> <div style="color: black;"><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrKp_pnn7u9BGauEPMed9LwXbshnogrTHYSzi2Ehc0F3AZJ0gLa6MVHi8gMgn4FbWleZp1RjUgjG9WIEFvtbEWmYSwsJxx6H-OIxQQQlx_j8CPko_vgGrqw3NbSFbii-s1O0GZeKzSbls/s1600/image-721351.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6892592385372805522" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrKp_pnn7u9BGauEPMed9LwXbshnogrTHYSzi2Ehc0F3AZJ0gLa6MVHi8gMgn4FbWleZp1RjUgjG9WIEFvtbEWmYSwsJxx6H-OIxQQQlx_j8CPko_vgGrqw3NbSFbii-s1O0GZeKzSbls/s320/image-721351.png" /></a><br /></font></div><p class="gmail-western" style="line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif" style="color: black;"><font size="4">Configure SMTP for Alerts</font></font></p><p class="gmail-western" style="line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif">Setup a simple local SMTP server which can only send emails from localhost. <br /><br /><font color="#0000ff">sudo apt install mailutils<br />sudo vim /etc/postfix/<a href="http://main.cf">main.cf</a></font></font></p><p class="gmail-western" style="line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif">Go to the End of the line,<br /><font color="#741b47">inet_interfaces = loopback-only<br />inet_protocols = ipv4</font><br /><br /><font color="#0000ff">sudo systemctl restart postfix</font><br /><br />Make sure your forward and reverse looks fine, otherwise it is very likely that email providers don't think this would be a valid email and you won't receive any emails. <br />Once verified, fire below command from the prometheus server,<br /><br /><font color="#000000">echo "This is the body" | mail -s "This is the subject" -a "FROM:admin@yourdomainname" your@email-address</font><br /><br />check your mail account, you would have received an email..</font></p><p class="gmail-western" style="line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif">configure the Alert Manager process to send emails when the alerting rules fire and resolve. <br /><font color="#0000ff">cd /etc/prometheus<br />cp alertmanager.yml alertmanager_orig.yml<br />cat > alertmanager.yml<br /></font>[ctrl-d]<br />Add the below contents and configure your alerts</font><font face="trebuchet ms, sans-serif" style="color: black;"><br /></font></p><p class="gmail-western" style="line-height: 115%; margin-bottom: 0.05cm;">route:</p><div style="line-height: 115%; margin-bottom: 0.05cm; text-align: left;"> receiver: smtp-local<br />receivers:<br /> - name: 'smtp-local'<br /> email_configs:<br /> - to: 'sunlnx@gmail.com'<br /> from: 'promoalertadmin@devtestlabs.in'<br /> require_tls: false<br /> #auth_username: 'alertmanager'<br /> #auth_password: 'password'<br /> #auth_secret: 'secret'<br /> #auth_identity: 'identity'<br /> smarthost: localhost:25<br /> send_resolved: true</div><p class="gmail-western" style="line-height: 115%; margin-bottom: 0.05cm;"><font face="trebuchet ms, sans-serif"></font></p><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt4QJJ1xpse798VOzgdyMBzqbH4iVK_eO0GsRsdD111zOiptT3ykOSUgr-eMTS2JmO7oab3twaXJ34qmIrAIuYVY0b3tResWA6zk-duBa_0rTd2B6wsbz7uaERTtkOAMM1CFhUm2B-YqQ/s1600/image-722831.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6892592393254681698" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt4QJJ1xpse798VOzgdyMBzqbH4iVK_eO0GsRsdD111zOiptT3ykOSUgr-eMTS2JmO7oab3twaXJ34qmIrAIuYVY0b3tResWA6zk-duBa_0rTd2B6wsbz7uaERTtkOAMM1CFhUm2B-YqQ/s320/image-722831.png" /></a></div></div><div class="gmail_default"><pre class="gmail-western" style="line-height: 115%; margin-bottom: 0.35cm;"><font face="trebuchet ms, sans-serif" style="color: black;"> <font color="#37474f"><font style="font-size: 9pt;">Now, you would have received an alert as the state is in Firing.</font></font> </font></pre><div style="color: black;"><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXrwQwf0Sq3E4AUlV0xcj5gzXFuHcxnYlOQeWox4pcuHTpTlplMgCHk1KLg4Mo8xyegZROfWl-ePAFSHt79Rdpx92iRbvTRhWkyhzvXGQ_1a2KZONi-uqLFdANgJFvVeUz7cVBYfcoB38/s1600/image-724319.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6892592397239140802" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXrwQwf0Sq3E4AUlV0xcj5gzXFuHcxnYlOQeWox4pcuHTpTlplMgCHk1KLg4Mo8xyegZROfWl-ePAFSHt79Rdpx92iRbvTRhWkyhzvXGQ_1a2KZONi-uqLFdANgJFvVeUz7cVBYfcoB38/s320/image-724319.png" /></a><br /></font></div><div style="color: black;"><font face="trebuchet ms, sans-serif"><br /></font></div><div style="color: black;"><div><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguGfDailBY4No67KLu4g6opUszfz4zNiAa_ZjZvlj_ELRrbsKK8fqQuB5qdVGMouNoH-0VfiypaFLfNomMp6IQF5G4AlQl0YJ8WAq_xPUXeYcj8qx5zZQpeLXsFRy0h4iN0zQN9Q06IqQ/s1600/image-725665.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6892592408014468578" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguGfDailBY4No67KLu4g6opUszfz4zNiAa_ZjZvlj_ELRrbsKK8fqQuB5qdVGMouNoH-0VfiypaFLfNomMp6IQF5G4AlQl0YJ8WAq_xPUXeYcj8qx5zZQpeLXsFRy0h4iN0zQN9Q06IqQ/s320/image-725665.png" /></a><br /></font></div></div><font face="trebuchet ms, sans-serif"><div class="gmail_default"><font face="trebuchet ms, sans-serif"><br /></font></div>Source mentioned in the email are w.r.t to localhost and we would configure it to use the prometheus source, to change it<br /><font color="#0000ff">sudo vim /etc/default/prometheus<br />ARGS="--web.enable-admin-api --web.external-url=<a href="https://example.com">https://example.com</a>"</font><br /><br />restart your prometheus server to take effect.</font></div><div class="gmail_default"><font face="trebuchet ms, sans-serif">systemctl restart prometheus</font></div><div class="gmail_default"><font face="trebuchet ms, sans-serif"><br /></font></div><div class="gmail_default"><font face="trebuchet ms, sans-serif">Thanks.</font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-54628045593970106582020-11-07T20:58:00.001+05:302020-11-08T10:21:50.290+05:30Prometheus Monitoring - Part 3<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Please check my previous posts as we have discussed in detail over scrape targets installations and configuration.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><a href="https://sunlnx.blogspot.com/2020/11/prometheus-monitoring-part-2.html">https://sunlnx.blogspot.com/2020/11/prometheus-monitoring-part-2.html</a><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">We would be discussing the below items on the Prometheus.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><ul><li><font face="trebuchet ms, sans-serif">PromQL Queries</font></li><li><font face="trebuchet ms, sans-serif">Saving/Recording Rules</font></li><li><font face="trebuchet ms, sans-serif">Alerting Rules</font></li></ul></div><div class="gmail_default" style="color:rgb(0,0,0)"><b style="font-family:"trebuchet ms",sans-serif"><font size="4">PromQL Queries</font></b><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><b style="font-family:"trebuchet ms",sans-serif"><font size="4"><br></font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">The query language used in Prometheus is called PromQL (Prometheus Query Language). The data can either be viewed as a graph, as tabled data, or in external systems such as Grafana, Zabbix and others.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br><b>Simple examples</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><b><br></b>node_cpu_seconds_total{}</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">node_cpu_seconds_total{instance="<a href="http://promonode01.devtestlabs.in:80">promonode01.devtestlabs.in:80</a>"} </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br><b>Regular Expressions</b><br>list only nodes with specific domains.<br><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">node_cpu_seconds_total{instance=~".*.devtestlabs.*"} </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">node_cpu_seconds_total{instance=~".*.devtestlabs.*",mode=~".*irq*"}</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br><b>Data Types</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><b><br>Scalar</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">A numeric floating point value</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br><b>Instant vector</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">A set of time series containing a single sample for each time series.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br><b>Range Vector</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">A set of time series containing a range of data points over time for each time series.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br>node_cpu_seconds_total{instance=~".*.devtestlabs.*",mode=~".*irq*"}[1m]<br>node_cpu_seconds_total{instance=~".*.devtestlabs.*",mode=~".*irq*"}[5m]<br><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><b>Functions/Subfunctions</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><ul><li><font face="trebuchet ms, sans-serif">Start with this instant vector node_netstat_Tcp_InSegs{instance="localhost:9100"}</font></li><li><font face="trebuchet ms, sans-serif">Convert it to a Range Vector and then convert it back to an instant vector using rate rate(node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m])</font></li><li><font face="trebuchet ms, sans-serif">Wrap it in the ceiling function ceil(rate(node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m]))</font></li><li><font face="trebuchet ms, sans-serif">Convert it to a range vector and get the per-second derivative of the time series deriv(ceil(rate(node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m]))[1m:])</font></li></ul></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">More Info: <a href="https://prometheus.io/docs/prometheus/latest/querying/functions/">https://prometheus.io/docs/prometheus/latest/querying/functions/</a><br><br><b><font size="4">Saving Rules</font></b><br>To run a query every time would be difficult, hence we can store the resultant query, Custom rules can be created and saved in the config file. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><b>Examples</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif">Memory available percentage: 100 - (100 * node_memory_MemFree_bytes / node_memory_MemTotal_bytes))<br>Root Disk Space: 100 * node_filesystem_free_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br>Login to your prometheus server and configure the rules as below</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><br><font color="#0000ff">vim /etc/prometheus/prometheus_rules.yml</font><br><font color="#741b47">groups:<br> - name: custom_rules<br> rules:<br> - record: node_memory_free_percent<br> expr: 100 - (100 * node_memory_MemFree_bytes / node_memory_MemTotal_bytes)<br> - record: node_filesystem_free_percent<br> expr: 100 * node_filesystem_free_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}</font><br><br><font color="#000000">check for syntax, </font><br><font color="#0000ff">promtool check rules prometheus_rules.yml</font><br><font color="#741b47">Checking prometheus_rules.yml<br> SUCCESS: 2 rules found</font><br><br><font color="#000000">Include these rules in the main </font>prometheus<font color="#000000"> configurations. </font><br><font color="#0000ff">vim /etc/prometheus/prometheus.yml</font><br><font color="#741b47">rule_files:<br> # - "first_rules.yml"<br> # - "second_rules.yml"<br> - "prometheus_rules.yml"</font><br><br>check<font color="#000000"> for the syntax of the main config file,</font></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><font color="#000000"><br></font><font color="#0000ff">promtool check config prometheus.yml</font><br><font color="#741b47">Checking prometheus.yml<br> SUCCESS: 1 rule files found<br><br>Checking prometheus_rules.yml<br> SUCCESS: 2 rules found</font><br><br><font color="#000000">since all are fine, restart </font>prometheus<font color="#000000"> server, </font><br><font color="#0000ff">systemctl restart prometheus</font><br><br><font color="#000000">You would now be able to see the custom rules in the dashboard. </font><br><font color="#000000"> </font></font></div><div><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiz77QgjK74Bfwwc58PKI3IhnZHcg3rkVvMin4DtVb5N6vOlF2bf2qPy0m2hfcxoQBL9S05PfB2kJDMO-muIHpDwSeWrxNefGF5xalcx3VxWS8aCo855VKJnVJq721aG2wsmSQ9kaq8OeA/s1600/image-738676.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiz77QgjK74Bfwwc58PKI3IhnZHcg3rkVvMin4DtVb5N6vOlF2bf2qPy0m2hfcxoQBL9S05PfB2kJDMO-muIHpDwSeWrxNefGF5xalcx3VxWS8aCo855VKJnVJq721aG2wsmSQ9kaq8OeA/s320/image-738676.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892404341439054946" /></a><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif" style=""><br><b style="color:rgb(0,0,0)"><font size="4">Alerts</font></b><br><font color="#000000">We will create a new group named alert_rules and add in the same config file which created earlier rules. </font><br><font color="#0000ff">vim /etc/promotheus/prometheus_rules.yml</font><br><font color="#741b47"> - name: alert_rules<br> rules:<br> - alert: InstanceDown<br> expr: up == 0<br> for: 1m<br> labels:<br> severity: critical<br> annotations:<br> summary: "Instance {{ $labels.instance }} down"<br> description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minute."<br><br> - alert: DiskSpaceFree10Percent<br> expr: node_filesystem_free_percent <= 10<br> labels:<br> severity: warning<br> annotations:<br> summary: "Instance {{ $labels.instance }} has 10% or less Free disk space"<br> description: "{{ $labels.instance }} has only {{ $value }}% or less free."</font><br><br><font color="#000000">check for the syntax of the config file and restart the service when there </font>are<font color="#000000"> SUCCESS message.</font><br><br><font color="#0000ff"> promtool check config prometheus.yml</font><br><font color="#741b47">Checking prometheus.yml<br> SUCCESS: 1 rule files found<br><br>Checking prometheus_rules.yml<br> SUCCESS: 3 rules found</font><br><br><font color="#0000ff">sudo service prometheus restart<br>sudo service prometheus status</font><br><font color="#000000"> </font><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyMsK4kaIj96Mh960DPbDP-Wf_19XJPpbSJkNmajIB4Pe8hFVvmP4K3WO-wTYeQFdFnU84F0Z9gd-UXrUsaXSAgJMP8zY1lr_EigiWeR4o2eB_zYMBLlrkplWSikXipWjC0lg45D0c3yA/s1600/image-740428.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyMsK4kaIj96Mh960DPbDP-Wf_19XJPpbSJkNmajIB4Pe8hFVvmP4K3WO-wTYeQFdFnU84F0Z9gd-UXrUsaXSAgJMP8zY1lr_EigiWeR4o2eB_zYMBLlrkplWSikXipWjC0lg45D0c3yA/s320/image-740428.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892404347142113218" /></a></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="trebuchet ms, sans-serif"><br></font></div><div><font face="trebuchet ms, sans-serif"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiko_AO3HnuI1CwVZkXwxsvPA8BkvJsPeBzXr1q71OIyddnG3UBEwOBFGooT7gL3TldT-ns-xUFl5skQrdYX-G7yUjwEQuW93x2dd9Aw9VjY24I98CQg4Sp0aCv-5FwnWODLr77MqWAFuc/s1600/image-741815.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiko_AO3HnuI1CwVZkXwxsvPA8BkvJsPeBzXr1q71OIyddnG3UBEwOBFGooT7gL3TldT-ns-xUFl5skQrdYX-G7yUjwEQuW93x2dd9Aw9VjY24I98CQg4Sp0aCv-5FwnWODLr77MqWAFuc/s320/image-741815.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892404355452974306" /></a><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-53573278459385735792020-11-07T16:43:00.000+05:302020-11-08T10:23:17.064+05:30Prometheus Monitoring - part 2<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif"><div style="color:rgb(0,0,0)">This is to further post on Prometheus monitoring. In the earlier section we have seen how to install and configure.<br><a href="https://sunlnx.blogspot.com/2020/11/prometheus-installations-and.html">https://sunlnx.blogspot.com/2020/11/prometheus-installations-and.html</a><br></div><div style="color:rgb(0,0,0)"><br></div><div style="color:rgb(0,0,0)">In this section, we could see below topics</div><div style="color:rgb(0,0,0)"><ul><li>Introduction to Scrape Targets</li><li>Adding a new node exporter externally to Promotheus node exporter</li><li>Configuring reverse proxy to the newly installed Prometheus node exporter</li><li>Deleting Time Service DataBase for unused nodes<br></li></ul></div><div style="color:rgb(0,0,0)"><b><font size="4">Introduction to Scrape Targets</font></b></div><div style="color:rgb(0,0,0)"><br></div><div style=""><font color="#000000">When you install prometheus, it would install two metric endpoints.</font><br><b style="color:rgb(0,0,0)">Prometheus </b><font color="#000000">: <a href="http://127.0.0.1:9090/metrics">http://127.0.0.1:9090/metrics</a></font><br><b style="color:rgb(0,0,0)">Node Exporter</b><font color="#000000"> : <a href="http://127.0.0.1:9100/metrics">http://127.0.0.1:9100/metrics</a></font><br><br><font color="#000000">You could see the same when you go to target sections in the Prometheus section.</font><br><br><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-drcz2eBNdUclemS2dJCtYmLLrRJC1zwoQ6fw_25e15DIRdk8slKNNiKxhyphenhyphentBmsinA9wJmE3yIShC3x3tpEt_sewflcXEncnc3SBVV94_ER-sRyhXmnZP6yrJ4360f47_KwtNaztEbTA/s1600/image-775245.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-drcz2eBNdUclemS2dJCtYmLLrRJC1zwoQ6fw_25e15DIRdk8slKNNiKxhyphenhyphentBmsinA9wJmE3yIShC3x3tpEt_sewflcXEncnc3SBVV94_ER-sRyhXmnZP6yrJ4360f47_KwtNaztEbTA/s320/image-775245.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339213414686786" /></a><br></div><font color="#000000"> </font><br><font color="#000000">when the above endpoints are executed in the local machine, you would be provided with all the metrics available on the prometheus as well as the nodes. </font><br><font color="#0000ff">curl <a href="http://127.0.0.1:9090/metrics">http://127.0.0.1:9090/metrics</a><br>curl <a href="http://127.0.0.1:9100/metrics">http://127.0.0.1:9100/metrics</a></font></div><div style=""><font color="#0000ff"><br></font></div><div style=""><font color="#000000">You can find the same in the UI when you go to the configuration section. </font></div><div style=""><font color="#000000"><br></font></div><div style=""><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCUyVUGmjoguGfscUKQwHPZhZb1bRNdpuwx7WSbpsWBIvSGa_RRv61_En6uHjB1bLZUoBXXTzX3RP0kokvjevSODlJ1Xvt0qBHvjX6nVhl_bEAa7IkQFmB5hhkOK60jJryakGsdan3ON8/s1600/image-777227.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCUyVUGmjoguGfscUKQwHPZhZb1bRNdpuwx7WSbpsWBIvSGa_RRv61_En6uHjB1bLZUoBXXTzX3RP0kokvjevSODlJ1Xvt0qBHvjX6nVhl_bEAa7IkQFmB5hhkOK60jJryakGsdan3ON8/s320/image-777227.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339222570343570" /></a><br></div></div><div style=""><font color="#0000ff"><br></font><font color="#000000">You would be able to scrape targets from the config file <b>/etc/prometheus/prometheus.yml</b></font><br><font color="#000000">Further nodes, could be added in scrape_confis and you can define the targets for specific intervals.</font><br><br><font color="#000000">If there are no </font>intervals<font color="#000000"> defined, it would be considered from the global section of the prometheus. i.e 15s. </font><br><br><font color="#0000ff">tail -20 /etc/prometheus/prometheus.yml</font><br><font color="#741b47"># Here it's Prometheus itself.<br>scrape_configs:<br> # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.<br> - job_name: 'prometheus'<br> # Override the global default and scrape targets from this job every 5 seconds.<br> scrape_interval: 5s<br> scrape_timeout: 5s<br> # metrics_path defaults to '/metrics'<br> # scheme defaults to 'http'.<br> static_configs:<br> - targets: ['localhost:9090']<br> - job_name: node<br> # If prometheus-node-exporter is installed, grab stats about the local<br> # machine by default.<br> static_configs:<br> - targets: ['localhost:9100']</font><br><font color="#000000"> </font><br></div><div style="color:rgb(0,0,0)">Search from the graph you could find many of the metrics related to the prometheus or with the nodes which you can configure. <br>search by "<b>prometheus</b>" or "<b>nodes</b>" will provide all the metrics details.<br> <br>Lets, look for the memory graph for the nodes... </div><div style="color:rgb(0,0,0)"><br>console </div><div style="color:rgb(0,0,0)"><br></div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigBPzbKsurg6dbcX2LiCGBI8d6pSgvntVoMhplXyMbLGaA1QOX73YrXuqtFBRBhwzU8FJFRka4azTa234Crqzr9zrFgDbTxHHRGGtpgytoJzcUkU5BKvLieUSZnKHmUPbZZba0Um-wJ4w/s1600/image-778565.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigBPzbKsurg6dbcX2LiCGBI8d6pSgvntVoMhplXyMbLGaA1QOX73YrXuqtFBRBhwzU8FJFRka4azTa234Crqzr9zrFgDbTxHHRGGtpgytoJzcUkU5BKvLieUSZnKHmUPbZZba0Um-wJ4w/s320/image-778565.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339230660292338" /></a><br></div><div style="color:rgb(0,0,0)">Graph <br> </div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxQPFhiwWi_uZ3DoJ3e260M74BsMZ7kkutRtRmN7pSn6leWiD6HLwTxkN9OQmUneBlwDDKgb1LbNKplw0quln559Y9KhU9o2cjRkc3zW6B169xUlnBM94YPI0V3A4xHlM-SpGYA5g_Ktg/s1600/image-780054.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxQPFhiwWi_uZ3DoJ3e260M74BsMZ7kkutRtRmN7pSn6leWiD6HLwTxkN9OQmUneBlwDDKgb1LbNKplw0quln559Y9KhU9o2cjRkc3zW6B169xUlnBM94YPI0V3A4xHlM-SpGYA5g_Ktg/s320/image-780054.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339233475538946" /></a><br></div><div style="color:rgb(0,0,0)"></div><div style="color:rgb(0,0,0)"><br></div><div style="color:rgb(0,0,0)"><b><font size="4">Install New node exporter</font></b></div><div style="color:rgb(0,0,0)"><font size="4"><b><br></b></font>Install an external Prometheus Node Exporter on a different server.</div><div style=""><br><font color="#0000ff">sudo apt-get update<br>apt install prometheus-node-exporter -y<br>sudo service prometheus-node-exporter status</font><br><br><font color="#000000">Register if you have any of your Domain hosting web </font>server<font color="#000000"> to point your IP to 'A'. </font></div><div style="color:rgb(0,0,0)"><br>verify your prometheus server is running fine - http://[your domain or ip]:9100/metrics</div><div style=""><br><font color="#000000">You can now block port 9100 externally, but leave it open internally for localhost </font><br><font color="#0000ff">iptables -A INPUT -p tcp -s localhost --dport 9100 -j ACCEPT<br>iptables -A INPUT -p tcp --dport 9100 -j DROP</font><br><br><font color="#000000">Even your prometheus server won't be able to query the new node as it would be </font>unreachable. <font color="#000000">there </font>are a couple<font color="#000000"> of solutions ..</font></div><div style=""><font color="#000000"><br>1) either you would allow from only your domain and restrict other traffic.</font><br><font color="#000000">2) You could install nginx reverse proxy and add a path specific for the node exporter.</font></div><div style=""><font color="#000000"><br>Add the new exporter to the prometheus server for monitoring. </font><br><font color="#0000ff">tail -6 /etc/prometheus/prometheus.yml</font><br><font color="#741b47"> - job_name: node_promonode01<br> #<br> static_configs:<br> - targets: ['localhost:9100']<br> - targets: ['<a href="http://promonode01.devtestlabs.in:9100">promonode01.devtestlabs.in:9100</a>']</font><br><br><font color="#000000">validate the configurations files, </font><br><font color="#000000"># </font><font color="#0000ff">promtool check config /etc/prometheus/prometheus.yml</font><br><font color="#000000">Checking /etc/prometheus/prometheus.yml</font><br><font color="#000000"> SUCCESS: 0 rule files found</font></div><div style=""><font color="#000000"><br>on successful, restart prometheus service.</font><br><font color="#0000ff">sudo service prometheus restart<br>sudo service prometheus status</font><br><font color="#000000"> </font></div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ3ACTYaqyDXKtO_iZAV-Q5q6_WE1WOl7LIJumFEW7q1cDSU_KZ3eyu8hnXzg6dV-N31LuHB37ak94FEvbO09fYQLDcnT2FuKt82pEbvx_qLonXByS0Rbh-LUI46qQ5cugLeCuZDZADrE/s1600/image-781351.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ3ACTYaqyDXKtO_iZAV-Q5q6_WE1WOl7LIJumFEW7q1cDSU_KZ3eyu8hnXzg6dV-N31LuHB37ak94FEvbO09fYQLDcnT2FuKt82pEbvx_qLonXByS0Rbh-LUI46qQ5cugLeCuZDZADrE/s320/image-781351.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339241500844130" /></a><br></div><div style="color:rgb(0,0,0)"><br><b><font size="4"> Configuring reverse proxy to the newly installed Prometheus node exporter</font></b></div><div style="color:rgb(0,0,0)"><font size="4"><b><br></b></font>Refer to previous post on how to setup nginx reverse proxy.. </div><div style="color:rgb(0,0,0)"><a href="https://sunlnx.blogspot.com/2020/11/prometheus-installations-and.html">https://sunlnx.blogspot.com/2020/11/prometheus-installations-and.html</a></div><div style=""><br><font color="#000000">Modify the nginx conf for the new node, with below location to allow prometheus server to query the data.</font><br><font color="#0000ff">sudo vim /etc/nginx/sites-enabled/promonode01</font><br><font color="#741b47"> location /metrics {<br> allow 206.189.139.255; <== Promotheus server<br> deny all;<br> proxy_pass <a href="http://localhost:9100/metrics">http://localhost:9100/metrics</a>;<br> }</font></div><div style=""><font color="#741b47"><br></font><font color="#000000">Even if there </font>is a request<font color="#000000"> http from the </font>prometheus<font color="#000000"> server, nginx would convert to https and will redirect traffic to provide metric details to the server. </font><br>These<font color="#000000"> configurations already </font>exist<font color="#000000"> on the server. </font></div><div style=""><br><font color="#741b47">server {<br> if ($host = <a href="http://promonode01.devtestlabs.in">promonode01.devtestlabs.in</a>) {<br> return 301 https://$host$request_uri;<br> } # managed by Certbot<br></font><br><font color="#000000">try to hit the URL, you would be getting the Forbidden Error as it won't allow your to get the metric details. </font><br><font color="#000000"> </font></div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT09HVb6mkyYhsGlkntPywUwPRAzPPHwPrRuqWJ8njH4q7ZIUkROk09kPsBfrTAbnMo8Ka5DUQLdWLKO43HrZGra3VS4w7BUvzyFK64wWfgfxkwZTv97BH0vKlA1_bvW4Iv9MFlI3roDg/s1600/image-782816.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT09HVb6mkyYhsGlkntPywUwPRAzPPHwPrRuqWJ8njH4q7ZIUkROk09kPsBfrTAbnMo8Ka5DUQLdWLKO43HrZGra3VS4w7BUvzyFK64wWfgfxkwZTv97BH0vKlA1_bvW4Iv9MFlI3roDg/s320/image-782816.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339248494123506" /></a><br></div><div style="color:rgb(0,0,0)"><br></div><div style=""><br><font color="#000000">Try to hit from the prometheus server using curl, you would be able to retrieve results. </font><br><b style="color:rgb(0,0,0)">curl -X GET <a href="https://promonode01.devtestlabs.in/metrics">https://promonode01.devtestlabs.in/metrics</a> </b><br><br><font color="#000000">Modify your </font>promotheus<font color="#000000"> server to scrape targets as we have configured reverse proxy</font><br><font color="#741b47"> - job_name: node_promonode01<br> #<br> static_configs:<br> - targets: ['localhost:9100']<br> - targets: ['<a href="http://promonode01.devtestlabs.in">promonode01.devtestlabs.in</a>']</font><br><br><font color="#000000">validate the config file and restart the </font>promotheus<font color="#000000"> server.</font><br><font color="#0000ff">service prometheus restart<br>service prometheus status</font><br><br><font color="#000000"> </font></div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxh2Z4ySbf9FAZlDK8LGtWU4tUSi0y9pqbSQS4ST_E6tUKm8Xx2QviZCyQHSoxiCekOeCFGjgbhzYk575JGiy49_dC3KM306hp8mnmKKD5vL43Sdh_0PylKD2Ajg5ghKmGVyYqKyWOM_I/s1600/image-784406.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxh2Z4ySbf9FAZlDK8LGtWU4tUSi0y9pqbSQS4ST_E6tUKm8Xx2QviZCyQHSoxiCekOeCFGjgbhzYk575JGiy49_dC3KM306hp8mnmKKD5vL43Sdh_0PylKD2Ajg5ghKmGVyYqKyWOM_I/s320/image-784406.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339253323188562" /></a><br></div><div style="color:rgb(0,0,0)"><br><b><font size="4"> Deleting Time Service DataBase for unused nodes</font></b></div><div style="color:rgb(0,0,0)"><font size="4"><b><br></b></font>Data will be automatically deleted after the storage retention time has passed. By default it is 15 days.<br>If you want to delete specific data earlier, then you are able. Since we have configured a couple of new job_name in the config files it would show us a couple of wanted nodes in the graphs, we would be deleting them. </div><div style="color:rgb(0,0,0)"><br>Even though below are the 3 nodes, from the time service database shows us a couple of more graphs...<br><br></div><div style="color:rgb(0,0,0)"><b>Console</b></div><div style="color:rgb(0,0,0)">go_threads{instance="localhost:9090",job="prometheus"} 9<br>go_threads{instance="localhost:9100",job="node"} 7<br>go_threads{instance="<a href="http://promonode01.devtestlabs.in:80">promonode01.devtestlabs.in:80</a>",job="node"} 7</div><div style="color:rgb(0,0,0)"><br><b>Graph</b><br> </div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnGJqsc-l9MVQXSFfWM0Op59lnGIw66ut2BugJNqcB2UwSLVZtAsftnRDMywmwJ8MUMjPMtN6aZtwaIYmi6DqiWaOwV9fF_7zb3rzV0wWt1tAUiwwL8BlBy-6wwrYtzGcLeo_wytN6omQ/s1600/image-786169.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnGJqsc-l9MVQXSFfWM0Op59lnGIw66ut2BugJNqcB2UwSLVZtAsftnRDMywmwJ8MUMjPMtN6aZtwaIYmi6DqiWaOwV9fF_7zb3rzV0wWt1tAUiwwL8BlBy-6wwrYtzGcLeo_wytN6omQ/s320/image-786169.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339260007785570" /></a><br></div><div style=""><font color="#000000">You need to enable the admin api in Prometheus before you can.</font><br><font color="#0000ff">sudo vim /etc/default/prometheus</font><br><font color="#741b47">ARGS="--web.enable-admin-api"</font><br><br><font color="#000000">Restart Prometheus and check status</font><br><font color="#0000ff">sudo service prometheus restart<br>sudo service prometheus status</font><br><br><font color="#000000">You can now make calls to the admin api.</font></div><div style=""><font color="#000000"><br></font><font color="#0000ff">curl -X POST \<br> -g '<a href="http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]={instance=">http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]={instance=</a>"localhost:9100",job="node_promonode01"}'<br><br>curl -X POST \<br> -g '<a href="http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]={instance=">http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]={instance=</a>"<a href="http://promonode01.devtestlabs.in:80">promonode01.devtestlabs.in:80</a>",job="node_promonode01"}'</font><br><font color="#000000"> </font></div><div style=""><font color="#000000">After executing the above queries from the prometheus server, you would no longer be able to see the nodes in the graphs.</font></div><div style="color:rgb(0,0,0)"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWblWmtvIFFWfZ3YDp6andf2D3Nt-XT2d88j3RTVBH-69MBIvOOzLCZfBLSfhWqFpDhY8TAZammmN-QB31I-2T6IimZ9sCFbVtbV7KnaptujIdvUYw8wBCrdNRxrKed7sP7SVtYYwDxp4/s1600/image-787606.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWblWmtvIFFWfZ3YDp6andf2D3Nt-XT2d88j3RTVBH-69MBIvOOzLCZfBLSfhWqFpDhY8TAZammmN-QB31I-2T6IimZ9sCFbVtbV7KnaptujIdvUYw8wBCrdNRxrKed7sP7SVtYYwDxp4/s320/image-787606.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6892339267497950082" /></a><br></div><div style=""><div style="color:rgb(0,0,0)"><br></div><font color="#000000">After deleting you should disable the admin api again,</font><br><font color="#0000ff">sudo nano /etc/default/prometheus</font><br><font color="#000000">Remove --web.enable-admin-api from the ARGS variable. eg,</font><br><br><font color="#000000">ARGS=""</font></div><div style=""><font color="#000000"><br>restart Prometheus and check status,</font><br><font color="#0000ff">sudo service prometheus restart<br>sudo service prometheus status</font><br><br>That's so far we have on this tutorial. we would see more on the coming tutorials.</div><div style=""><br></div><div style=""><br><br></div></div></div></div></div></div></div></div></div></div></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-57338970625613937402020-11-06T19:42:00.001+05:302020-11-08T16:45:34.550+05:30Prometheus Monitoring - part 1<div dir="ltr"><div class="gmail_default" style="color: black; font-family: "trebuchet ms", sans-serif;"><b><font size="4">Prometheus Monitoring </font></b><br />We shall be installing and configuring the Prometheus on the Ubuntu server. you can either create a local box or from any of the cloud providers.</div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><br /></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><b>Arch Diagram</b></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFBXDehBqFIGJtydXG3DSC-0-5ErwXMfOZXj5qk0QNQVM2TRcWgtZgzvnn3UNS6gw_eHQ1Ty36oJPzo0Kth1NIhpeMnqlekD8yPbaLI-ARlO71EofhXzcWwUiCuGUwy4CiLHLRXoSyrYM/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="434" data-original-width="684" height="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFBXDehBqFIGJtydXG3DSC-0-5ErwXMfOZXj5qk0QNQVM2TRcWgtZgzvnn3UNS6gw_eHQ1Ty36oJPzo0Kth1NIhpeMnqlekD8yPbaLI-ARlO71EofhXzcWwUiCuGUwy4CiLHLRXoSyrYM/" width="320" /></a></div><br /><br /></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font size="4" style="color: black;"><b>Prerequisites</b></font><br /><font color="#000000">I have set up this monitoring tool using Ubuntu 20.04 LTS Server with root access. You can use other operating systems, such as Centos, but since it was already installed for some demo purpose I wanted to configure this. </font><br /><br /><b style="color: black;"><font size="4">Installations</font></b><br /><font color="#000000">Prometheus package installed both </font><b style="color: black;">Prometheus </b><font color="#000000">and the </font><b style="color: black;">Prometheus Node</b><font color="#000000"> to be installed. </font><br /><br /><font color="#0000ff">sudo apt-get update<br />sudo apt install prometheus -y<br />sudo service prometheus status<br />sudo service prometheus-node-exporter status</font><br /><br /><font color="#000000">Prometheus should now be running.</font><br /><font color="#0000ff">ps -u prometheus</font><br /><br /><font color="#000000">You can visit it at http://[your ip address]:9090</font><br /><font size="4"><br /><font color="#000000"><b>Pointing your 'A' Domain name </b></font></font><br /><font color="#000000">If your Prometheus server is accessible from the internet, you want it to look more professional to clients, login to your domain name provider, and add an A Name record that points to the IP address </font>of the new<font color="#000000"> Prometheus server.</font><br /><br /><font color="#000000" size="4"><b>Reverse Proxy Prometheus with Nginx</b></font><br /><font color="#000000">One option to help secure our Prometheus server is to put it behind a reverse proxy so that we can later add SSL and an Authentication layer over the default unrestricted Prometheus web interface.</font><br /><br /><font color="#0000ff">sudo apt install nginx -y</font><br /><font color="#0000ff">sudo vim /etc/nginx/sites-enabled/prometheus</font><br /><br /><font color="#741b47">server {<br /> listen 80;<br /> listen [::]:80;<br /> server_name prometheus.YOUR-DOMAIN-NAME;<br /><br /> location / {<br /> proxy_pass <a href="http://localhost:9090/">http://localhost:9090/</a>;<br /> }<br />}</font><br /><br /><font color="#000000">Save and test the new configuration has no errors</font><br /><font color="#000000">nginx -t</font><br /><br /><font color="#000000"><a href="http://YOUR-DOMAIN-NAME">http://YOUR-DOMAIN-NAME</a></font><br /><font color="#000000">Visiting your ip address directly will still show the default Nginx welcome page. you can remove </font><br /><br /><font color="#000000">rm /etc/nginx/sites-enabled/default</font><br /><br /><font color="#000000">restart nginx, </font><br /><font color="#0000ff">sudo service nginx restart<br />sudo service nginx status</font><br /><br /><font color="#000000" size="4"><b>Add SSL to Prometheus Reverse Proxy</b></font><br /><font color="#000000">We will now add transport encryption to the Prometheus web user interface.</font><br /><font color="#000000">Certbot will install a LetsEncrypt SSL certificate for free. Ensure your domain name has propagated before running CertBot.</font><br /><br /><font color="#0000ff">sudo snap install --classic certbot<br />sudo certbot --nginx</font><br /><br /><snip></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;">.</div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;">.<br /><font color="#000000">Follow the prompts and select the domain name I want to secure.</font><br />.</div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;">.</div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><br /><font color="#000000">- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -</font><br /><font color="#000000">Congratulations! You have successfully enabled <a href="https://prometheus.YOUR-DOMAIN-NAME">https://prometheus.YOUR-DOMAIN-NAME</a></font><br /><font color="#000000">- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -</font><br /><snip><br /><br /><br /><font color="#000000" size="4"><b>Add Basic User Authentication Prometheus UI</b></font><br /><font color="#000000">Everything is great so far, but anybody in the world with the internet access and the URL can visit my Prometheus server and see my data.</font><br /><font color="#000000">To solve this problem, we will add user authentication.</font><br /><br /><font color="#0000ff">cd /etc/nginx<br />sudo apt install apache2-utils<br />htpasswd -c /etc/nginx/.htpasswd admin</font><br /><br /><font color="#000000">Nginx Prometheus config file, </font><br /><font color="#000000"><br /></font></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font color="#0000ff">sudo vim /etc/nginx/sites-enabled/prometheus</font><br /><font color="#741b47">server {<br /> ...<br /><br /> #addition authentication properties<br /> auth_basic "Protected Area"; <=============== append<br /> auth_basic_user_file /etc/nginx/.htpasswd; <=== append<br /><br /> location / {<br /> proxy_pass <a href="http://localhost:9090/">http://localhost:9090/</a>;<br /> }<br /><br /> ...<br />}</font><br /><br /><font color="#000000">restart nginx, </font><br /><font color="#0000ff">sudo service nginx restart<br />sudo service nginx status</font></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font color="#0000ff"><br /></font></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font color="#000000">when you try to open your Prometheus server, it would prompt for your basic authentication.<br /></font><br /><font color="#000000">you would still be able to access the IP:9090 of the Prometheus server and hence we block ports from external connections.</font><br /><font color="#000000"><br /></font></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font color="#0000ff">iptables -A INPUT -p tcp -s localhost --dport 9090 -j ACCEPT<br />iptables -A INPUT -p tcp --dport 9090 -j DROP<br />iptables -A INPUT -p tcp -s localhost --dport 9100 -j ACCEPT<br />iptables -A INPUT -p tcp --dport 9100 -j DROP<br />iptables -L</font><br /><br /><font color="#000000">To save rules permanently, </font><br /><font color="#0000ff">sudo apt install iptables-persistent<br />iptables-save > /etc/iptables/rules.v4<br />iptables-save > /etc/iptables/rules.v6</font></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font color="#0000ff"><br /></font></div><div class="gmail_default" style="font-family: "trebuchet ms", sans-serif;"><font color="#000000">you have now successfully installed Prometheus server on your machine.</font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-82922899984043162802020-08-03T09:13:00.001+05:302020-11-08T10:23:36.724+05:30Monitor PR for merge into master using TeamCity<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">You would have an automated build using teamcity, however we could use an PR and once they are manually verified and approved we could proceed to merge for master. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Modify the branch specifications <b>+:refs/pull/*/merge</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJzZK-hXD6R7XKQAqVUb1xAEbo5tKAN1MOuL04Bwy6NFCICt3oBPbKhzY4UGjnLB2a6CUbqwDjW5Dltf_jEQjq-dC3hPk12_qXfmonalQleAgljSK6EGZ_vBZvLOJg6vyROVVZDtLjjsg/s1600/image-734199.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJzZK-hXD6R7XKQAqVUb1xAEbo5tKAN1MOuL04Bwy6NFCICt3oBPbKhzY4UGjnLB2a6CUbqwDjW5Dltf_jEQjq-dC3hPk12_qXfmonalQleAgljSK6EGZ_vBZvLOJg6vyROVVZDtLjjsg/s320/image-734199.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856598468849050946" /></a><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Once you add the some file in the "feature" branch, it would create an build as we are watching this build in the teamcity. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)">You need to edit the Build Feature and modify the below settings to ensure you have PR for the the build to merge master branch</div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3dNEqXWzxWrz-Ul5vRom9YkIvIXeBYf1nqBBYpSMN9BBlPZFfYFtW309A68FxLGf2TmXVNv8BI1DSG18OfZffwTfHcajbgmBDiSvGMMlag6nGFN97-t2QdIY1dxDCKnK2-mJYmIlOXEk/s1600/image-735340.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3dNEqXWzxWrz-Ul5vRom9YkIvIXeBYf1nqBBYpSMN9BBlPZFfYFtW309A68FxLGf2TmXVNv8BI1DSG18OfZffwTfHcajbgmBDiSvGMMlag6nGFN97-t2QdIY1dxDCKnK2-mJYmIlOXEk/s320/image-735340.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856598473907966818" /></a><br></div><div><br></div><div>Once you create a new file in the feature branch, create PR and wait for the build to succeed in the feature branch. </div><div>Once the build is successful it will perform checks for the PR and once the changes are fine fo you, you can merge to master. </div><div><br></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6adu6RNDwxq2iSjm-7Nt1hpw2GYioZSBqxMp95scEykPKowpbAxSIRFtIhWr2GqvqfGgUgKGytKzc9Zb0Vlc4HFZZXInO9nRi7hy5AKmcail3hDK5xpoX9ogG0KUxMOBTDUV0OQXltz4/s1600/image-736245.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6adu6RNDwxq2iSjm-7Nt1hpw2GYioZSBqxMp95scEykPKowpbAxSIRFtIhWr2GqvqfGgUgKGytKzc9Zb0Vlc4HFZZXInO9nRi7hy5AKmcail3hDK5xpoX9ogG0KUxMOBTDUV0OQXltz4/s320/image-736245.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856598476284729794" /></a><br></div></div><div><br></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghTlIN6XZ1-UCNgvjmpcwPd7457qShzNnA9NFqZQEVlOf77ssg1DFMFHal-ha-Bu8n4q5xavcMWUWWVJfB3_INBK5HICwW2i2wtUOcPmVIpkAgqhlsYVCaulkJAobGIUCXEPICRzTZ2JY/s1600/image-737191.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghTlIN6XZ1-UCNgvjmpcwPd7457qShzNnA9NFqZQEVlOf77ssg1DFMFHal-ha-Bu8n4q5xavcMWUWWVJfB3_INBK5HICwW2i2wtUOcPmVIpkAgqhlsYVCaulkJAobGIUCXEPICRzTZ2JY/s320/image-737191.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856598482311218226" /></a><br></div></div><div>So you have now all the changes from the branch to be in master. Once you have approved the changes, it will start to build to master.</div><div><br></div><div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWhNazIh8q3fQzaZzBLSo7dopdyje00HLhQ_V219ogfKQBzeu8fvs10jBJK3wnfxEsO8ZNiZOxHiSpnq5HRSPXhe24UmTL3p-SgyrtM6-xKBnxYEMMZ8eksRtMst8SAj3-T0qf3JqwCNk/s1600/image-737833.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWhNazIh8q3fQzaZzBLSo7dopdyje00HLhQ_V219ogfKQBzeu8fvs10jBJK3wnfxEsO8ZNiZOxHiSpnq5HRSPXhe24UmTL3p-SgyrtM6-xKBnxYEMMZ8eksRtMst8SAj3-T0qf3JqwCNk/s320/image-737833.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856598485656642002" /></a><br></div></div><div><br></div><div>This is how you can configure teamcity to have an manual approach for merging into master branch.</div><div><br></div></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com1tag:blogger.com,1999:blog-4652559897711604839.post-83694547168166435192020-08-03T08:38:00.001+05:302020-11-08T10:23:42.811+05:30AutoMerge Branch to Master using TeamCity<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Create a new VCS repository providing github. I have configured the repository in the root directory of the VCS in the teamcity project. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><table class="gmail-projectHierarchy" id="gmail-adminOverview" style="border-collapse:collapse;border-spacing:0px;width:1309px;color:rgb(21,21,21);font-size:13.12px"><tbody><tr class="gmail-project gmail-tr-depth-0" style="background:none;border-bottom:1px solid rgb(238,238,238)"><td class="gmail-project_name gmail-depth-0" style="padding-top:0.3em;padding-bottom:0.3em;padding-left:2px"><font face="monospace"><a href="http://192.168.56.120:8111/admin/editProject.html?init=1&projectId=_Root" class="gmail-icon_before gmail-icon16 gmail-project-icon" style="color:rgb(21,100,194);text-decoration-line:none"><Root project></a> <span class="gmail-project_description" style="padding-left:5px;color:rgb(136,136,136)">Contains all other projects</span></font></td><td class="edit gmail-highlight" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><a href="http://192.168.56.120:8111/admin/editProject.html?init=1&projectId=_Root" style="color:rgb(21,100,194);text-decoration-line:none"><font face="monospace">Edit</font></a></td><td class="edit" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"></td></tr><tr><td class="gmail-project_content_holder" colspan="10" style="padding:0px;border:none;border-collapse:collapse"><div id="gmail-adminOverview_project_handle__RootContent" class="gmail-refreshable" style="margin:0px;padding:0px"><div id="gmail-adminOverview_project_handle__RootContentInner" class="gmail-refreshableInner" style="margin:0px;padding:0px"><table class="gmail-project_content_holder" id="gmail-btbadminOverview_project_handle__Root" style="border-collapse:collapse;border-spacing:0px;padding:0px;margin:0px;border:none;width:1309px"><tbody><tr><td id="gmail-adminOverview_project_handle__Root_subproj_holder" colspan="10" class="gmail-sub_project_table" style="padding:0px;border:none"><table class="gmail-projectHierarchy" id="gmail-adminOverview" style="border-collapse:collapse;border-spacing:0px;width:1309px"><tbody><tr id="gmail-adminOverview_project_handle_project3" class="gmail-project gmail-tr-depth-1" style="background:none;border-bottom:1px solid rgb(238,238,238)"><td class="gmail-project_name gmail-depth-1" style="padding-top:0.3em;padding-bottom:0.3em;padding-left:20px"><font face="monospace"><span title="Click to collapse/expand details" class="gmail-icon gmail-icon16 gmail-handle gmail-handle_collapsed" id="gmail-blockHandleadminOverview_project_handle_project3" style="vertical-align:text-bottom;display:inline-block;background-color:transparent;background-image:url("/css/../webComponents/teamcity-elements/icons/teamcity-sprite.png");background-repeat:no-repeat;background-size:240px 532px;width:16px;height:16px;outline:none;background-position:-173px -16px"></span> <a href="http://192.168.56.120:8111/admin/editProject.html?init=1&projectId=JavaExamples" class="gmail-icon_before gmail-icon16 gmail-project-icon" style="color:rgb(21,100,194);text-decoration-line:none">JavaExamples</a> <span class="gmail-project_description" style="padding-left:5px;color:rgb(136,136,136)">Java Examples for build and deploy artifacts</span></font></td><td class="edit gmail-highlight" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><a href="http://192.168.56.120:8111/admin/editProject.html?init=1&projectId=JavaExamples" style="color:rgb(21,100,194);text-decoration-line:none"><font face="monospace">Edit</font></a></td><td class="edit" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><span class="gmail-btn-group" id="gmail-sp_span_prjActionsJavaExamples" style="display:block"><button class="gmail-btn gmail-btn_mini gmail-popupLink gmail-nonInitializedActionPopup" type="button" style="font-size:13px;padding-top:0px;padding-bottom:0px;outline:0px;background-color:rgb(246,246,246);color:rgb(51,51,51);border-width:1px;border-style:solid;border-color:rgb(204,204,204) rgb(204,204,204) rgb(187,187,187);border-radius:4px;white-space:nowrap"><font face="monospace"><span class="gmail-icon-list-ul" style="text-decoration:inherit;display:inline;width:auto;height:auto;line-height:normal;vertical-align:baseline;background-image:none;background-position:0% 0%;background-repeat:repeat;margin-top:0px;padding-right:4px;padding-left:2px;color:rgb(128,128,128);font-size:10.4px"></span> <span class="gmail-icon-caret-down" id="gmail-prjActionsJavaExamples" style="text-decoration:inherit;display:inline;width:auto;height:auto;line-height:normal;vertical-align:baseline;background-image:none;background-position:0% 0%;background-repeat:repeat;margin-top:0px;color:rgb(128,128,128)"></span></font></button></span></td></tr><tr><td class="gmail-project_content_holder" colspan="10" style="padding:0px;border:none;border-collapse:collapse"><div id="gmail-adminOverview_project_handle_project3Content" class="gmail-refreshable" style="margin:0px;padding:0px"><div id="gmail-adminOverview_project_handle_project3ContentInner" class="gmail-refreshableInner" style="margin:0px;padding:0px"></div></div></td></tr><tr id="gmail-adminOverview_project_handle_project5" class="gmail-project gmail-tr-depth-1" style="background:none rgb(249,249,249);border-bottom:1px solid rgb(238,238,238)"><td class="gmail-project_name gmail-depth-1" style="padding-top:0.3em;padding-bottom:0.3em;padding-left:20px"><font face="monospace"><span title="Click to collapse/expand details" class="gmail-icon gmail-icon16 gmail-handle gmail-handle_expanded" id="gmail-blockHandleadminOverview_project_handle_project5" style="vertical-align:text-bottom;display:inline-block;background-color:transparent;background-image:url("/css/../webComponents/teamcity-elements/icons/teamcity-sprite.png");background-repeat:no-repeat;background-size:240px 532px;width:16px;height:16px;outline:none;background-position:-175px 4px"></span> <a href="http://192.168.56.120:8111/admin/editProject.html?init=1&projectId=SpringRestExampleStandaloneRepo" class="gmail-icon_before gmail-icon16 gmail-project-icon" style="color:rgb(21,100,194)">Spring Rest Example Standalone Repo</a><span class="gmail-project_description" style="padding-left:5px;color:rgb(136,136,136)"></span></font></td><td class="edit gmail-highlight" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><a href="http://192.168.56.120:8111/admin/editProject.html?init=1&projectId=SpringRestExampleStandaloneRepo" style="color:rgb(21,100,194);text-decoration-line:none"><font face="monospace">Edit</font></a></td><td class="edit" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><span class="gmail-btn-group" id="gmail-sp_span_prjActionsSpringRestExampleStandaloneRepo" style="display:block"><button class="gmail-btn gmail-btn_mini gmail-popupLink gmail-nonInitializedActionPopup" type="button" style="font-size:13px;padding-top:0px;padding-bottom:0px;outline:0px;background-color:rgb(246,246,246);color:rgb(51,51,51);border-width:1px;border-style:solid;border-color:rgb(204,204,204) rgb(204,204,204) rgb(187,187,187);border-radius:4px;white-space:nowrap"><font face="monospace"><span class="gmail-icon-list-ul" style="text-decoration:inherit;display:inline;width:auto;height:auto;line-height:normal;vertical-align:baseline;background-image:none;background-position:0% 0%;background-repeat:repeat;margin-top:0px;padding-right:4px;padding-left:2px;color:rgb(128,128,128);font-size:10.4px"></span> <span class="gmail-icon-caret-down" id="gmail-prjActionsSpringRestExampleStandaloneRepo" style="text-decoration:inherit;display:inline;width:auto;height:auto;line-height:normal;vertical-align:baseline;background-image:none;background-position:0% 0%;background-repeat:repeat;margin-top:0px;color:rgb(128,128,128)"></span></font></button></span></td></tr><tr><td class="gmail-project_content_holder" colspan="10" style="padding:0px;border:none;border-collapse:collapse"><div id="gmail-adminOverview_project_handle_project5Content" class="gmail-refreshable" style="margin:0px;padding:0px"><div id="gmail-adminOverview_project_handle_project5ContentInner" class="gmail-refreshableInner" style="margin:0px;padding:0px"><table class="gmail-project_content_holder" id="gmail-btbadminOverview_project_handle_project5" style="border-collapse:collapse;border-spacing:0px;padding:0px;margin:0px;border:none;width:1309px"><tbody><tr class="gmail-build_type gmail-tr-depth-3" style="border-bottom:1px solid rgb(238,238,238)"><td class="gmail-build_type_name gmail-depth-3" style="padding-top:0.3em;padding-bottom:0.3em;padding-left:60px"><font face="monospace"><font color="#cc0000" style="text-decoration-line:none"><a href="http://192.168.56.120:8111/admin/editBuild.html?init=1&id=buildType:SpringRestExampleStandaloneRepo_SpringRestExampleStandaloneRepo&cameFromUrl=%2Fadmin%2Fadmin.html%3Fitem%3Dprojects" class="gmail-icon_before gmail-icon16 gmail-buildType-icon" style="text-decoration-line:none"><span class="gmail_default" style="text-decoration-line:none;font-weight:bold;color:rgb(0,0,0)"></span><b style="text-decoration-line:none">Spr</b><span class="gmail_default" style="text-decoration-line:none;font-weight:bold"></span><b style="text-decoration-line:none">ing Rest Example Standalone Repo</b></a><span class="gmail_default" style="text-decoration-line:none;color:rgb(0,0,0)"><b><a href="http://192.168.56.120:8111/admin/editBuild.html?init=1&id=buildType:SpringRestExampleStandaloneRepo_SpringRestExampleStandaloneRepo&cameFromUrl=%2Fadmin%2Fadmin.html%3Fitem%3Dprojects" class="gmail-icon_before gmail-icon16 gmail-buildType-icon" style="text-decoration-line:none"> -> this repo needs to be set for any change in the branch has to merged to master if build successful</a><br></b><a href="http://192.168.56.120:8111/admin/editBuild.html?init=1&id=buildType:SpringRestExampleStandaloneRepo_SpringRestExampleStandaloneRepo&cameFromUrl=%2Fadmin%2Fadmin.html%3Fitem%3Dprojects" class="gmail-icon_before gmail-icon16 gmail-buildType-icon"><br></a></span></font><span class="gmail-build_type_description" style="padding-left:5px;color:rgb(136,136,136)"></span></font></td><td class="edit gmail-highlight" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><span class="gmail-pc" id="editControlbt8" style="padding:1px 2px 2px 0px;white-space:nowrap"><font face="monospace"><span class="gmail-pc__label"><a href="http://192.168.56.120:8111/admin/editBuild.html?init=1&id=buildType:SpringRestExampleStandaloneRepo_SpringRestExampleStandaloneRepo&cameFromUrl=%2Fadmin%2Fadmin.html%3Fitem%3Dprojects" title="Edit build configuration settings" class="gmail-nonInitializedBuildTypeMenu" style="color:rgb(21,100,194);text-decoration-line:none">Edit</a> </span><span class="gmail-pc__toggle-wrapper"> <span class="gmail-icon gmail-icon16 gmail-toggle" style="vertical-align:text-bottom;display:inline-block;background-color:transparent;background-image:url("/css/../webComponents/teamcity-elements/icons/teamcity-sprite.png");background-repeat:no-repeat;background-size:240px 532px;width:18px;height:18px;outline:none;background-position:-140px -140px"></span></span></font></span></td><td class="edit" style="padding-top:0.3em;padding-bottom:0.3em;width:60px;vertical-align:top"><span class="gmail-btn-group" id="gmail-sp_span_btActionsbt8" style="display:block"><button class="gmail-btn gmail-btn_mini gmail-popupLink gmail-nonInitializedActionPopup" type="button" style="font-size:13px;padding-top:0px;padding-bottom:0px;outline:0px;background-color:rgb(246,246,246);color:rgb(51,51,51);border-width:1px;border-style:solid;border-color:rgb(204,204,204) rgb(204,204,204) rgb(187,187,187);border-radius:4px;white-space:nowrap"><font face="monospace"><span class="gmail-icon-list-ul" style="text-decoration:inherit;display:inline;width:auto;height:auto;line-height:normal;vertical-align:baseline;background-image:none;background-position:0% 0%;background-repeat:repeat;margin-top:0px;padding-right:4px;padding-left:2px;color:rgb(128,128,128);font-size:10.4px"></span> <span class="gmail-icon-caret-down" id="gmail-btActionsbt8" style="text-decoration:inherit;display:inline;width:auto;height:auto;line-height:normal;vertical-align:baseline;background-image:none;background-position:0% 0%;background-repeat:repeat;margin-top:0px;color:rgb(128,128,128)"></span></font></button></span></td></tr></tbody></table></div></div></td></tr></tbody></table></td></tr></tbody></table></div></div></td></tr></tbody></table><div class="gmail_default" style="color:rgb(0,0,0)"><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMGMjUE9Rqm9Elaxlv_YqpoYrlHQCsZwDYyvL4h9MxbYIKExuOOym0oKol6eE4THEW8KnEadsdZlLiKDPEBB3NwDJeDo9FI7VeodFtS5FvM_-VVu2unO0bxaawO3FcmX0ABJnCUCPRYNg/s1600/image-798899.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMGMjUE9Rqm9Elaxlv_YqpoYrlHQCsZwDYyvL4h9MxbYIKExuOOym0oKol6eE4THEW8KnEadsdZlLiKDPEBB3NwDJeDo9FI7VeodFtS5FvM_-VVu2unO0bxaawO3FcmX0ABJnCUCPRYNg/s320/image-798899.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856589297714241234" /></a><br></div><div><br></div></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Created a new branch "feature" and would be modifying the code, once the changes are merged and build is successful then it would automatically merge to the master branch.</font><span style="font-family:monospace"></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Modify the VCS in the "Branch Specification" as<b> +:refs/heads/*</b> which lets the teamcity to check for branches other than the master. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQAUPUteH-4g481g76VtyLZ6xO5iLu3EIEK6ce0VjXpUXOCGq0l_1e0e-tyxRQvKNvcqisagQUK0n4f5800RHzhk1xyLl63yPc4c5iHps2WhRAKLCQQdyD0awqdPgAq9eDDvSGgUSalfY/s1600/image-700819.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQAUPUteH-4g481g76VtyLZ6xO5iLu3EIEK6ce0VjXpUXOCGq0l_1e0e-tyxRQvKNvcqisagQUK0n4f5800RHzhk1xyLl63yPc4c5iHps2WhRAKLCQQdyD0awqdPgAq9eDDvSGgUSalfY/s320/image-700819.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856589304377148754" /></a><br></div></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Select "Triggers" and select for auto trigger from the VCS.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Select "Build Features" and select "Automerge changes" in watch build in branch </font><b style="font-family:monospace">+:feature</b><span style="font-family:monospace"> when new build is created.</span></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="font-family:monospace"><br></span></div><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeX-7E34_LD8NxMQx0cxqgUJ3dqxsUXbwGRqvi_9CP3WppQsR_lyQF_I6FY34RM08qy47V4rY8J6FMKtWRYEFf1rl2ACRTS5AdkDzQ-fgiTAf2ST1BDlOM2tQONSrlghNkKhHdoupt4fY/s1600/image-701721.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeX-7E34_LD8NxMQx0cxqgUJ3dqxsUXbwGRqvi_9CP3WppQsR_lyQF_I6FY34RM08qy47V4rY8J6FMKtWRYEFf1rl2ACRTS5AdkDzQ-fgiTAf2ST1BDlOM2tQONSrlghNkKhHdoupt4fY/s320/image-701721.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856589312384412066" /></a><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="font-family:monospace"><br></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="font-family:monospace">Ensure, you have an agent being attached to the VCS to execute the build.</span><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG2eJq8pdjIWm8wRnb32Y5SRdUc-g9zvzXsx9WuDrUl97fEeU5wg_sVPF2mFYxvFMoxaPwDI4582opvKzcRLSWRrDP9EjkEZww6mNVNPaunphoyB763XYk310oXOPVRoLVjqhmiYNVK_w/s1600/image-702737.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG2eJq8pdjIWm8wRnb32Y5SRdUc-g9zvzXsx9WuDrUl97fEeU5wg_sVPF2mFYxvFMoxaPwDI4582opvKzcRLSWRrDP9EjkEZww6mNVNPaunphoyB763XYk310oXOPVRoLVjqhmiYNVK_w/s320/image-702737.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6856589316508034226" /></a><br></div></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif;color:rgb(0,0,0)"><br></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-21534666427224063072020-08-02T10:46:00.001+05:302020-11-08T10:23:49.662+05:30TeamCity Server and BulidAgent installations on Centos 7<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><span style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:700;font-stretch:inherit;line-height:inherit;vertical-align:baseline;color:rgb(34,49,63)">TeamCity</span><span style="color:rgb(34,49,63)"> is a Java-based build management and continuous integration server from JetBrains. </span><span style="color:rgb(34,49,63)"> </span></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><span style="color:rgb(34,49,63)">A Freemium license for up to 20 build configurations and 3 free Build Agent licenses is available.</span></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(34,49,63)"><font face="monospace"><br></font></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><h3 style="margin:0px 0px 25px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:1.3;vertical-align:baseline;color:rgb(34,49,63)"><font size="4" face="monospace">Prerequisites</font></h3><div><font face="monospace">- Centos 7 to be installed.</font></div><div><font face="monospace">- Java to be installed </font></div></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">- postgresql to be installed. </font></div><div class="gmail_default" style="color:rgb(0,0,0)"><b><font face="monospace"><br></font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4" face="monospace"><b>Java Installations</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"># sudo yum install java-1.8.0-openjdk<br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4" face="monospace"><b>Postgresql Installations</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br>Install postgresql<br># sudo yum install postgresql-server postgresql-contrib<br><br>Initialize the database<br># sudo postgresql-setup initdb<br><br>Start the database<br># sudo systemctl start postgresql<br><br>Enable postgresql during boot<br># sudo systemctl enable postgresql<br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4" face="monospace"><b>TeamCity Installations</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Download Teamcity tar archive from official website</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">mkdir /opt/teamcity<br>cd /opt/teamcity<br>wget <a href="https://download.jetbrains.com/teamcity/TeamCity-2020.1.2.tar.gz" target="_blank">https://download.jetbrains.com/teamcity/TeamCity-2020.1.2.tar.gz</a><br>tar -xzvf TeamCity-2020.1.2.tar.gz<br><br><b>Download PostgreSQL JDBC driver</b><br>We need to download PostgreSQL JDBC driver in order to use PostgreSQL database for TeamCity.<br><br>mkdir -p /opt/teamcity/TeamCity/.BuildServer/lib/jdbc/<br>wget <a href="https://jdbc.postgresql.org/download/postgresql-42.2.14.jar" target="_blank">https://jdbc.postgresql.org/download/postgresql-42.2.14.jar</a> -P /opt/teamcity/TeamCity/.BuildServer/lib/jdbc/</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b><br></b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b>Start service</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"># cd /opt/teamcity/TeamCity/bin</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"># ./startup.sh</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b>Stop Service</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"># ./shutdown.sh</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b>Restart Service</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"># ./shutdown.sh && ./startup.sh</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b><br></b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b>Configure postgresql</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">sudo su - postgres</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">vim data/<span style="color:rgb(34,49,63);font-size:14px">pg_hba.conf</span></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><span style="color:rgb(34,49,63);font-size:14px"><br></span></font></div><font face="monospace"><span class="gmail_default" style="color:rgb(0,0,0)">Modify to md5 at the end for these below lines</span><br class="gmail-Apple-interchange-newline"># "local" is for Unix domain socket connections only<br>local all all <b><span class="gmail_default" style="color:rgb(0,0,0)"></span>md5</b><br># IPv4 local connections:<br>host all all <a href="http://127.0.0.1/32">127.0.0.1/32</a> <b><span class="gmail_default" style="color:rgb(0,0,0)"></span>md5</b><br># IPv6 local connections:<br>host all all ::1/128 <b><span class="gmail_default" style="color:rgb(0,0,0)"></span>md5</b><br></font><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"># exit</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"># systemctl restart postgresql</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"># psql -U postgres</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">postgres=# CREATE USER teamcity WITH PASSWORD 'teamcity';<br>CREATE ROLE<br>postgres=# CREATE DATABASE teamcity OWNER teamcity;<br>CREATE DATABASE<br>postgres=# \q<br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b><br></b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace" size="4"><b>TeamCity Web</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">point your browser to teamcity Server IP address and follow these below steps</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Step 1: Change the Data DirectoryHere<br>we will change the data directory to <b>/opt/teamcity/TeamCity/.BuildServer</b><br><br>Step 2: Now we have to setup database connection<br><br>Database host[:port] - <b>localhost</b><br>Database name: <b>xxxxxx</b><br>User name: <b>xxxxxx</b><br>Password: <b>xxxxx</b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><b><br></b>Step 3: Accept licence</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br>Step 4: Create Admin account <br><br>TeamCity installations has been completed. <br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><b><font size="4" face="monospace"><br></font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><b><font size="4" face="monospace">TeamCity BuildAgent Installations</font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">Download Zipfile</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><span style="color:rgb(34,49,63);font-size:14px">We need to download a zip file from the teamcity server to install it on the agent. Replace the server-url with your server ip or server hostname.</span><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(34,49,63);font-size:14px"><font face="monospace"><br></font></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(34,49,63);font-size:14px"><font face="monospace">mkdir /opt/teamcity</font></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">wget <a href="http://server-ip:8111/update/buildAgent.zip">http://server-ip:8111/update/buildAgent.zip</a><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">sudo unzip buildAgent.zip<br>sudo chmod +x bin/agent.sh</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">vim buildAgent.dist.properties</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">serverUrl=<a href="http://server-ip:8111/">http://server-ip:8111/</a><br>cp conf/buildAgent.dist.properties conf/buildAgent.properties<span style="color:rgb(34,49,63);font-size:14px"><br></span></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><b><font face="monospace">Start Service</font></b></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"># cd /opt/teamcity/buildAgent/bin</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"># ./agent.sh</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace">You would need to authorize the agent in the buildserver once its get detected. you can now start your build and work along.</font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-955889235764125432020-01-12T15:42:00.002+05:302020-11-06T20:32:22.448+05:30Create Kubernetes Cluster in AWS using kops<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;">You can create k8's cluster using kops which will spin your k8s cluster within 5-7 mins. </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;">This would create EC2 instances with your required number of master and worker nodes, joins them to the cluster and you can continue deploying your application.<br />
<br />
Let's start with Prerequisite<br />
Ensure you have already installed below binaries..<br />
<br />
1. Kubernetes<br />
2. Kops<br />
3. aws-cli tools <br />
<br />
Create an IAM user and ensure he has "Administrator" policy attached to his profile. <br />
From your local workstation, execute below commands for validations..<br />
<br />
<b><span style="font-size: medium;">Prerequisite</span></b><br />
<br />
samperay@master:~$ aws iam list-users<br />
{<br />
"Users": [<br />
{<br />
"Path": "/",<br />
"UserName": "samperay",<br />
"UserId": " ",<br />
"Arn": "arn:aws: ",<br />
"CreateDate": " ",<br />
"PasswordLastUsed": " "<br />
}<br />
]<br />
}<br />
samperay@master:~$ <br />
<br />
samperay@master:~$ kubectl version <br />
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:20:10Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}<br />
samperay@master:~$<br />
<br />
samperay@master:~$ kops version<br />
Version 1.10.0 (git-8b52ea6d1)<br />
samperay@master:~$ <br />
<br />
<b><span style="font-size: medium;">Create Cluster</span></b><br />
<br />
samperay@master:~$ kops create cluster \<br />
> --state "s3://<a href="http://k8master.k8s.local.com/" target="_blank">k8master.k8s.local.com</a>" \<br />
> --zones "ap-south-1a" \<br />
> --master-count 1 \<br />
> --master-size=t2.micro \<br />
> --node-count 1 \<br />
> --node-size=t2.micro \<br />
> --name=k8master.k8s.local \<br />
> --yes<br />
I0112 09:58:45.726120 10182 create_cluster.go:480] Inferred --cloud=aws from zone "ap-south-1a"<br />
I0112 09:58:45.981370 10182 subnets.go:184] Assigned CIDR <a href="http://172.20.32.0/19" target="_blank">172.20.32.0/19</a> to subnet ap-south-1a<br />
I0112 09:58:46.668579 10182 create_cluster.go:1351] Using SSH public key: /home/samperay/.ssh/id_rsa.pub<br />
<br />
*********************************************************************************<br />
<br />
A new kops version is available: 1.11.1<br />
<br />
Upgrading is recommended<br />
More information: <a href="https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.11.1" target="_blank">https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.11.1</a><br />
<br />
*********************************************************************************<br />
<br />
I0112 09:58:49.750925 10182 apply_cluster.go:505] Gossip DNS: skipping DNS validation<br />
I0112 09:58:50.328520 10182 executor.go:103] Tasks: 0 done / 77 total; 30 can run<br />
I0112 09:58:51.233293 10182 vfs_castore.go:735] Issuing new certificate: "apiserver-aggregator-ca"<br />
I0112 09:58:51.331223 10182 vfs_castore.go:735] Issuing new certificate: "ca"<br />
I0112 09:58:52.809148 10182 executor.go:103] Tasks: 30 done / 77 total; 24 can run<br />
I0112 09:58:53.627921 10182 vfs_castore.go:735] Issuing new certificate: "kubelet"<br />
I0112 09:58:53.828622 10182 vfs_castore.go:735] Issuing new certificate: "kops"<br />
I0112 09:58:53.917293 10182 vfs_castore.go:735] Issuing new certificate: "apiserver-aggregator"<br />
I0112 09:58:53.935965 10182 vfs_castore.go:735] Issuing new certificate: "kube-proxy"<br />
I0112 09:58:54.044695 10182 vfs_castore.go:735] Issuing new certificate: "apiserver-proxy-client"<br />
I0112 09:58:54.139700 10182 vfs_castore.go:735] Issuing new certificate: "kubecfg"<br />
I0112 09:58:54.157747 10182 vfs_castore.go:735] Issuing new certificate: "kube-controller-manager"<br />
I0112 09:58:54.219260 10182 vfs_castore.go:735] Issuing new certificate: "kubelet-api"<br />
I0112 09:58:54.432620 10182 vfs_castore.go:735] Issuing new certificate: "kube-scheduler"<br />
I0112 09:58:54.942804 10182 executor.go:103] Tasks: 54 done / 77 total; 19 can run<br />
I0112 09:58:55.586592 10182 launchconfiguration.go:380] waiting for IAM instance profile "nodes.k8master.k8s.local" to be ready<br />
I0112 09:58:55.673860 10182 launchconfiguration.go:380] waiting for IAM instance profile "masters.k8master.k8s.local" to be ready<br />
I0112 09:59:06.221535 10182 executor.go:103] Tasks: 73 done / 77 total; 3 can run<br />
I0112 09:59:07.267706 10182 vfs_castore.go:735] Issuing new certificate: "master"<br />
I0112 09:59:07.766924 10182 executor.go:103] Tasks: 76 done / 77 total; 1 can run<br />
I0112 09:59:08.197751 10182 executor.go:103] Tasks: 77 done / 77 total; 0 can run<br />
I0112 09:59:09.038744 10182 update_cluster.go:290] Exporting kubecfg for cluster<br />
kops has set your kubectl context to k8master.k8s.local<br />
<br />
Cluster is starting. It should be ready in a few minutes.<br />
<br />
Suggestions:<br />
* validate cluster: kops validate cluster<br />
* list nodes: kubectl get nodes --show-labels<br />
* ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.k8master.k8s.local<br />
* the admin user is specific to Debian. If not using Debian please use the appropriate user based on your OS.<br />
* read about installing addons at: <a href="https://github.com/kubernetes/kops/blob/master/docs/addons.md" target="_blank">https://github.com/kubernetes/kops/blob/master/docs/addons.md</a>.<br />
<br />
samperay@master:~$ <br />
<br />
It would take around 5 mins to spin the instances, create cluster and join the client. validate your cluster. Once the cluster status shows ready you have completed your cluster build.</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
</span></div>
<div class="gmail_default" style="color: black;">
<div>
<span style="font-family: monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbpAVfOo4tLjIwziQHtWJcfZ5eKIUXzf_BKdtnF4-zb5VmzGk2ZQeCmbEDwFFd4ooVsNshgkvOXWZn9UwnVSOaHZQs0bgE0Anr5HQMAhXjQMdpQji5orA1OKp8u_pC97zl0Hs8gUrW30M/s1600/Screenshot+from+2020-01-12+10-18-09-737764.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6780997181351047762" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbpAVfOo4tLjIwziQHtWJcfZ5eKIUXzf_BKdtnF4-zb5VmzGk2ZQeCmbEDwFFd4ooVsNshgkvOXWZn9UwnVSOaHZQs0bgE0Anr5HQMAhXjQMdpQji5orA1OKp8u_pC97zl0Hs8gUrW30M/s320/Screenshot+from+2020-01-12+10-18-09-737764.png" /></a></span></div>
</div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><b><span style="font-size: medium;">Validate Cluster Status</span></b><br />
<br />
samperay@master:~$ kops validate cluster --state "s3://<a href="http://k8master.k8s.local.com/" target="_blank">k8master.k8s.local.com</a>" --name=k8master.k8s.local <br />
Validating cluster k8master.k8s.local<br />
<br />
INSTANCE GROUPS<br />
NAME ROLE MACHINETYPE MIN MAX SUBNETS<br />
master-ap-south-1a Master t2.micro 1 1 ap-south-1a<br />
nodes Node t2.micro 1 1 ap-south-1a<br />
<br />
NODE STATUS<br />
NAME ROLE READY<br />
ip-172-20-45-131.ap-south-1.compute.internal node True<br />
ip-172-20-54-84.ap-south-1.compute.internal master True<br />
<br />
Your cluster k8master.k8s.local is ready<br />
samperay@master:~$ <br />
<br />
verify is your cluster listing the nodes. <br />
<br />
samperay@master:~$ kubectl get nodes<br />
NAME STATUS ROLES AGE VERSION<br />
ip-172-20-45-131.ap-south-1.compute.internal Ready node 5m v1.11.10<br />
ip-172-20-54-84.ap-south-1.compute.internal Ready master 6m v1.11.10<br />
samperay@master:~$ <br />
<br />
<b><span style="font-size: medium;">Testing</span></b><br />
<br />
Create a deployment for nginx and start deploying in containers</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
samperay@master:~$ cat nginx_deployment.yml <br />
apiVersion: apps/v1<br />
kind: Deployment<br />
metadata:<br />
name: nginx-deployment<br />
labels:<br />
app: nginx<br />
spec:<br />
replicas: 3<br />
selector:<br />
matchLabels:<br />
app: nginx<br />
template:<br />
metadata:<br />
labels:<br />
app: nginx<br />
spec:<br />
containers:<br />
- name: nginx<br />
image: nginx:1.7.9<br />
ports:<br />
- containerPort: 80<br />
samperay@master:~$ <br />
<br />
samperay@master:~$ kubectl apply -f nginx_deployment.yml <br />
deployment.apps/nginx-deployment created<br />
samperay@master:~$ <br />
<br />
Create a service definition using Loadbalancer as its in the cloud platform and then try accessing it.<br />
<br />
samperay@master:~$ cat nginx_service.yml <br />
kind: Service<br />
apiVersion: v1<br />
<br />
metadata:<br />
name: nginx-elb<br />
namespace: default<br />
annotations:<br />
<a href="http://service.beta.kubernetes.io/aws-load-balancer-type" target="_blank">service.beta.kubernetes.io/aws-load-balancer-type</a>: "nlb"<br />
<br />
spec:<br />
type: LoadBalancer<br />
selector:<br />
app: nginx<br />
ports:<br />
- name: http<br />
port: 80<br />
targetPort: 80<br />
samperay@master:~$ <br />
<br />
samperay@master:~$ kubectl create -f nginx_service.yml <br />
service/nginx-elb created<br />
samperay@master:~$ </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
</span></div>
<div class="gmail_default" style="color: black;">
<div>
<span style="font-family: monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjydDl3PkK-6WgKhyphenhyphenwdY_B4QV3L1kJ_UuvKLNGz2sFrI2QZ9uOn-5GvptSMNj1Se9xnkt4vqx6mfCL3F2_ugtF3EnQt3F6EgzzXO8MDmFyaEs9VwgAK6ugVL2j1mHxrPEbz5KbVnymoh0/s1600/Screenshot+from+2020-01-12+10-18-54-738737.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6780997188003303362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjydDl3PkK-6WgKhyphenhyphenwdY_B4QV3L1kJ_UuvKLNGz2sFrI2QZ9uOn-5GvptSMNj1Se9xnkt4vqx6mfCL3F2_ugtF3EnQt3F6EgzzXO8MDmFyaEs9VwgAK6ugVL2j1mHxrPEbz5KbVnymoh0/s320/Screenshot+from+2020-01-12+10-18-54-738737.png" /></a></span></div>
<span style="font-family: monospace;"></span><br />
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><span style="font-family: monospace;"><br />
</span></span></div>
<div class="gmail_default" style="color: black;">
<div>
<span style="font-family: monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmGDY3Pkgy1y4CeYQap0wDnx_r-o5YU0I1TTrqBfn_J0AycEfPMntE7iqhejv50r84Uyiaw0LAvWmathuAu4AzVPTaKUCjDZfbkTtXNe2-G5rd_Z42u7AXzIcw3hkUfvYSyC3PkDt_6A8/s1600/Screenshot+from+2020-01-12+10-19-53-739389.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6780997187963904930" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmGDY3Pkgy1y4CeYQap0wDnx_r-o5YU0I1TTrqBfn_J0AycEfPMntE7iqhejv50r84Uyiaw0LAvWmathuAu4AzVPTaKUCjDZfbkTtXNe2-G5rd_Z42u7AXzIcw3hkUfvYSyC3PkDt_6A8/s320/Screenshot+from+2020-01-12+10-19-53-739389.png" /></a></span></div>
</div>
<span style="font-family: monospace;"><br />
<b><span style="font-size: medium;">Delete Cluster</span></b><br />
<br />
First, delete you application which are scheduled on the pods. </span></div>
<div class="gmail_default">
<span style="font-family: monospace;"><span style="color: black;">Removing service and deployment files ..</span></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;">samperay@master:~$ kubectl delete -f nginx_service.yml <br />
service "nginx-elb" deleted<br />
samperay@master:~$ </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
samperay@master:~$ kubectl delete -f nginx_deployment.yml <br />
deployment.apps "nginx-deployment" deleted<br />
samperay@master:~$ <br />
<br />
samperay@master:~$ kops delete cluster --state "s3://<a href="http://k8master.k8s.local.com/" target="_blank">k8master.k8s.local.com</a>" --name=k8master.k8s.local --yes<br />
TYPE NAME ID<br />
autoscaling-config master-ap-south-1a.masters.k8master.k8s.local-20200112042855 master-ap-south-1a.masters.k8master.k8s.local-20200112042855<br />
autoscaling-config nodes.k8master.k8s.local-20200112042855 nodes.k8master.k8s.local-20200112042855<br />
autoscaling-group master-ap-south-1a.masters.k8master.k8s.local master-ap-south-1a.masters.k8master.k8s.local<br />
autoscaling-group nodes.k8master.k8s.local nodes.k8master.k8s.local<br />
dhcp-options k8master.k8s.local dopt-053e74d7bc2e4e103<br />
iam-instance-profile masters.k8master.k8s.local masters.k8master.k8s.local<br />
iam-instance-profile nodes.k8master.k8s.local nodes.k8master.k8s.local<br />
iam-role masters.k8master.k8s.local masters.k8master.k8s.local<br />
iam-role nodes.k8master.k8s.local nodes.k8master.k8s.local<br />
instance master-ap-south-1a.masters.k8master.k8s.local i-02e581bdd00018208<br />
instance nodes.k8master.k8s.local i-00a96bdc8a9634372<br />
internet-gateway k8master.k8s.local igw-05fbf90230d26402f<br />
keypair kubernetes.k8master.k8s.local-22:db:4b:99:62:32:46:6c:d5:07:6a:10:a3:77:41:f4 kubernetes.k8master.k8s.local-22:db:4b:99:62:32:46:6c:d5:07:6a:10:a3:77:41:f4<br />
load-balancer api.k8master.k8s.local api-k8master-k8s-local-81d239<br />
route-table k8master.k8s.local rtb-0be624a42f3a50e73<br />
security-group api-elb.k8master.k8s.local sg-0c297322154723471<br />
security-group masters.k8master.k8s.local sg-02a2332aefd024a2a<br />
security-group nodes.k8master.k8s.local sg-03070d1f2b649bd50<br />
subnet ap-south-1a.k8master.k8s.local subnet-0a20150af0ede199a<br />
volume a.etcd-events.k8master.k8s.local vol-0139f37a67c7fcba9<br />
volume a.etcd-main.k8master.k8s.local vol-0ffba47a560b655ec<br />
vpc k8master.k8s.local vpc-01b43c6c68e8d8720<br />
<br />
load-balancer:api-k8master-k8s-local-81d239 ok<br />
keypair:kubernetes.k8master.k8s.local-22:db:4b:99:62:32:46:6c:d5:07:6a:10:a3:77:41:f4 ok<br />
autoscaling-group:master-ap-south-1a.masters.k8master.k8s.local ok<br />
instance:i-00a96bdc8a9634372 ok<br />
instance:i-02e581bdd00018208 ok<br />
autoscaling-group:nodes.k8master.k8s.local ok<br />
internet-gateway:igw-05fbf90230d26402f still has dependencies, will retry<br />
iam-instance-profile:nodes.k8master.k8s.local ok<br />
iam-instance-profile:masters.k8master.k8s.local ok<br />
iam-role:masters.k8master.k8s.local ok<br />
iam-role:nodes.k8master.k8s.local ok<br />
volume:vol-0139f37a67c7fcba9 still has dependencies, will retry<br />
autoscaling-config:nodes.k8master.k8s.local-20200112042855 ok<br />
autoscaling-config:master-ap-south-1a.masters.k8master.k8s.local-20200112042855 ok<br />
volume:vol-0ffba47a560b655ec still has dependencies, will retry<br />
subnet:subnet-0a20150af0ede199a still has dependencies, will retry<br />
security-group:sg-0c297322154723471 still has dependencies, will retry<br />
security-group:sg-03070d1f2b649bd50 still has dependencies, will retry<br />
security-group:sg-02a2332aefd024a2a still has dependencies, will retry<br />
Not all resources deleted; waiting before reattempting deletion<br />
route-table:rtb-0be624a42f3a50e73<br />
vpc:vpc-01b43c6c68e8d8720<br />
security-group:sg-02a2332aefd024a2a<br />
security-group:sg-0c297322154723471<br />
dhcp-options:dopt-053e74d7bc2e4e103<br />
volume:vol-0ffba47a560b655ec<br />
volume:vol-0139f37a67c7fcba9<br />
security-group:sg-03070d1f2b649bd50<br />
subnet:subnet-0a20150af0ede199a<br />
internet-gateway:igw-05fbf90230d26402f<br />
subnet:subnet-0a20150af0ede199a still has dependencies, will retry<br />
security-group:sg-03070d1f2b649bd50 still has dependencies, will retry<br />
volume:vol-0139f37a67c7fcba9 still has dependencies, will retry<br />
volume:vol-0ffba47a560b655ec still has dependencies, will retry<br />
internet-gateway:igw-05fbf90230d26402f still has dependencies, will retry<br />
security-group:sg-02a2332aefd024a2a still has dependencies, will retry<br />
security-group:sg-0c297322154723471 ok<br />
Not all resources deleted; waiting before reattempting deletion<br />
security-group:sg-03070d1f2b649bd50<br />
subnet:subnet-0a20150af0ede199a<br />
internet-gateway:igw-05fbf90230d26402f<br />
volume:vol-0ffba47a560b655ec<br />
volume:vol-0139f37a67c7fcba9<br />
vpc:vpc-01b43c6c68e8d8720<br />
security-group:sg-02a2332aefd024a2a<br />
route-table:rtb-0be624a42f3a50e73<br />
dhcp-options:dopt-053e74d7bc2e4e103<br />
subnet:subnet-0a20150af0ede199a still has dependencies, will retry<br />
volume:vol-0139f37a67c7fcba9 ok<br />
volume:vol-0ffba47a560b655ec ok<br />
internet-gateway:igw-05fbf90230d26402f still has dependencies, will retry<br />
security-group:sg-03070d1f2b649bd50 still has dependencies, will retry<br />
security-group:sg-02a2332aefd024a2a still has dependencies, will retry<br />
Not all resources deleted; waiting before reattempting deletion<br />
vpc:vpc-01b43c6c68e8d8720<br />
security-group:sg-02a2332aefd024a2a<br />
route-table:rtb-0be624a42f3a50e73<br />
dhcp-options:dopt-053e74d7bc2e4e103<br />
security-group:sg-03070d1f2b649bd50<br />
subnet:subnet-0a20150af0ede199a<br />
internet-gateway:igw-05fbf90230d26402f<br />
security-group:sg-02a2332aefd024a2a ok<br />
subnet:subnet-0a20150af0ede199a ok<br />
security-group:sg-03070d1f2b649bd50 ok<br />
internet-gateway:igw-05fbf90230d26402f ok<br />
route-table:rtb-0be624a42f3a50e73 ok<br />
vpc:vpc-01b43c6c68e8d8720 ok<br />
dhcp-options:dopt-053e74d7bc2e4e103 ok<br />
Deleted kubectl config for k8master.k8s.local<br />
Deleted cluster: "k8master.k8s.local"<br />
samperay@master:~$ <br />
<br />
Now, its completed, </span><span style="font-family: monospace;">Feel free to share !</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br />
</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;">Thanks</span></div>
<div class="gmail_default" style="color: black;">
</div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-61005111487483107522020-01-05T11:04:00.001+05:302020-11-06T20:32:30.952+05:30Upgrading kubeadm clusters from v1.16 to v1.17 - Part 3<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;">since, this is the latest upgrade at the time of writing, I am upgrading cluster from v1.16 to v1.17. <br />I am using only 1 master and 1 node so I would cordon the master before I proceed so that any new pods won't be scheduled on the master as there is un-tained the nodes because of my resource crunch. <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.16.4<br />node01 Ready <none> 215d v1.16.4<br />k8@master:~$ <br />k8@master:~$ kubectl cordon master<br />node/master cordoned<br />k8@master:~$ <br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.16.4<br />node01 Ready <none> 215d v1.16.4<br />k8@master:~$ <br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.17.0-00 && sudo apt-mark hold kubeadm<br />Canceled hold on kubeadm.<br />Hit:1 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:3 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubeadm<br />1 upgraded, 0 newly installed, 0 to remove and 225 not upgraded.<br />Need to get 8,059 kB of archives.<br />After this operation, 4,911 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.17.0-00 [8,059 kB]<br />Fetched 8,059 kB in 2s (4,530 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubeadm_1.17.0-00_amd64.deb ...<br />Unpacking kubeadm (1.17.0-00) over (1.16.4-00) ...<br />Setting up kubeadm (1.17.0-00) ...<br />kubeadm set on hold.<br />k8@master:~$ <br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:17:50Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$ <br />k8@master:~$<br /><br />k8@master:~$ sudo kubeadm upgrade plan <br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade] Fetching available versions to upgrade to<br />[upgrade/versions] Cluster version: v1.16.4<br />[upgrade/versions] kubeadm version: v1.17.0<br />[upgrade/versions] Latest stable version: v1.17.0<br />[upgrade/versions] Latest version in the v1.16 series: v1.16.4<br /><br />Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':<br />COMPONENT CURRENT AVAILABLE<br />Kubelet 2 x v1.16.4 v1.17.0<br /><br />Upgrade to the latest stable version:<br /><br />COMPONENT CURRENT AVAILABLE<br />API Server v1.16.4 v1.17.0<br />Controller Manager v1.16.4 v1.17.0<br />Scheduler v1.16.4 v1.17.0<br />Kube Proxy v1.16.4 v1.17.0<br />CoreDNS 1.6.2 1.6.5<br />Etcd 3.3.15 3.4.3-0<br /><br />You can now apply the upgrade by executing the following command:<br /><br /> kubeadm upgrade apply v1.17.0<br /><br />_____________________________________________________________________<br /><br />k8@master:~$ sudo kubeadm upgrade apply v1.17.0<br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade/version] You have chosen to change the cluster version to "v1.17.0"<br />[upgrade/versions] Cluster version: v1.16.4<br />[upgrade/versions] kubeadm version: v1.17.0<br />[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y<br />[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]<br />[upgrade/prepull] Prepulling image for component etcd.<br />[upgrade/prepull] Prepulling image for component kube-apiserver.<br />[upgrade/prepull] Prepulling image for component kube-controller-manager.<br />[upgrade/prepull] Prepulling image for component kube-scheduler.<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[upgrade/prepull] Prepulled image for component kube-scheduler.<br />[upgrade/prepull] Prepulled image for component kube-controller-manager.<br />[upgrade/prepull] Prepulled image for component etcd.<br />[upgrade/prepull] Prepulled image for component kube-apiserver.<br />[upgrade/prepull] Successfully prepulled the images for all the control plane components<br />[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.17.0"...<br />Static pod: kube-apiserver-master hash: 35f32b612a788851c3a8a4d9a66d3763<br />Static pod: kube-controller-manager-master hash: 02d086a9d02511e1fab232604d81ae74<br />Static pod: kube-scheduler-master hash: 732be3f14f79b5c85c2b9fc7df90d045<br />[upgrade/etcd] Upgrading to TLS for etcd<br />Static pod: etcd-master hash: 2651e1682591cc3914e2ee74a2a9e2dc<br />[upgrade/staticpods] Preparing for "etcd" upgrade<br />[upgrade/staticpods] Renewing etcd-server certificate<br />[upgrade/staticpods] Renewing etcd-peer certificate<br />[upgrade/staticpods] Renewing etcd-healthcheck-client certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/etcd.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-05-05-11-56/etcd.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: etcd-master hash: 2651e1682591cc3914e2ee74a2a9e2dc<br />Static pod: etcd-master hash: 2651e1682591cc3914e2ee74a2a9e2dc<br />Static pod: etcd-master hash: 2651e1682591cc3914e2ee74a2a9e2dc<br />Static pod: etcd-master hash: 2651e1682591cc3914e2ee74a2a9e2dc<br />Static pod: etcd-master hash: e21bda8353bb262054f042c2d851ea41<br />[apiclient] Found 1 Pods for label selector component=etcd<br />[upgrade/staticpods] Component "etcd" upgraded successfully!<br />[upgrade/etcd] Waiting for etcd to become available<br />[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests298337124"<br />W0105 05:13:19.416462 8997 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"<br />[upgrade/staticpods] Preparing for "kube-apiserver" upgrade<br />[upgrade/staticpods] Renewing apiserver certificate<br />[upgrade/staticpods] Renewing apiserver-kubelet-client certificate<br />[upgrade/staticpods] Renewing front-proxy-client certificate<br />[upgrade/staticpods] Renewing apiserver-etcd-client certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-05-05-11-56/kube-apiserver.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-apiserver-master hash: 35f32b612a788851c3a8a4d9a66d3763<br />Static pod: kube-apiserver-master hash: 35f32b612a788851c3a8a4d9a66d3763<br />Static pod: kube-apiserver-master hash: 35f32b612a788851c3a8a4d9a66d3763<br />Static pod: kube-apiserver-master hash: 8a168e41d705499409dd6586a3ac846d<br />[apiclient] Found 1 Pods for label selector component=kube-apiserver<br />[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade<br />[upgrade/staticpods] Renewing controller-manager.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-05-05-11-56/kube-controller-manager.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-controller-manager-master hash: 02d086a9d02511e1fab232604d81ae74<br />Static pod: kube-controller-manager-master hash: 02d086a9d02511e1fab232604d81ae74<br />Static pod: kube-controller-manager-master hash: 02d086a9d02511e1fab232604d81ae74<br />Static pod: kube-controller-manager-master hash: 02d086a9d02511e1fab232604d81ae74<br />Static pod: kube-controller-manager-master hash: 341d082c6764ae10963a30dd95004c2a<br />[apiclient] Found 1 Pods for label selector component=kube-controller-manager<br />[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-scheduler" upgrade<br />[upgrade/staticpods] Renewing scheduler.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-05-05-11-56/kube-scheduler.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-scheduler-master hash: 732be3f14f79b5c85c2b9fc7df90d045<br />Static pod: kube-scheduler-master hash: ff67867321338ffd885039e188f6b424<br />[apiclient] Found 1 Pods for label selector component=kube-scheduler<br />[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!<br />[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace<br />[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster<br />[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials<br />[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token<br />[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster<br />[addons]: Migrating CoreDNS Corefile<br />[addons] Applied essential addon: CoreDNS<br />[addons] Applied essential addon: kube-proxy<br />[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.17.0". Enjoy!<br /><br />[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.<br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.17.0-00 kubectl=1.17.0-00 && sudo apt-mark hold kubelet kubectl<br />Canceled hold on kubelet.<br />Canceled hold on kubectl.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease <br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:4 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease<br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 223 not upgraded.<br />Need to get 27.9 MB of archives.<br />After this operation, 14.8 MB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.17.0-00 [8,742 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.17.0-00 [19.2 MB]<br />Fetched 27.9 MB in 4s (7,345 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.17.0-00_amd64.deb ...<br />Unpacking kubectl (1.17.0-00) over (1.16.4-00) ...<br />Preparing to unpack .../kubelet_1.17.0-00_amd64.deb ...<br />Unpacking kubelet (1.17.0-00) over (1.16.4-00) ...<br />Setting up kubelet (1.17.0-00) ...<br />Setting up kubectl (1.17.0-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@master:~$ <br />k8@master:~$ sudo systemctl restart kubelet<br />k8@master:~$<br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.17.0 <= Kubernetes cluster upgraded<br />node01 Ready <none> 215d v1.16.4<br />k8@master:~$<br /><br />k8@master:~$ kubectl uncordon master<br />node/master uncordoned<br />k8@master:~$<br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.17.0<br />node01 Ready <none> 215d v1.16.4<br />k8@master:~$ <br /><br />+++++++++++++++= upgrading worker nodes +++++++++++++++ <br /><br />k8@master:~$ kubectl drain node01 --ignore-daemonsets<br />node/node01 cordoned<br />k8@master:~$<br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 216d v1.17.0<br />node01 Ready,SchedulingDisabled <none> 215d v1.16.4<br />k8@master:~$ <br /><br />k8@node01:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.4", GitCommit:"224be7bdce5a9dd0c2fd0d46b83865648e2fe0ba", GitTreeState:"clean", BuildDate:"2019-12-11T12:44:45Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}<br />k8@node01:~$ <br />k8@node01:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.17.0-00 && sudo apt-mark hold kubeadm<br />Canceled hold on kubeadm.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease<br />Hit:3 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease<br />Hit:4 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease<br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubeadm<br />1 upgraded, 0 newly installed, 0 to remove and 168 not upgraded.<br />Need to get 8,059 kB of archives.<br />After this operation, 4,911 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.17.0-00 [8,059 kB]<br />Fetched 8,059 kB in 3s (2,346 kB/s) <br />(Reading database ... 41117 files and directories currently installed.)<br />Preparing to unpack .../kubeadm_1.17.0-00_amd64.deb ...<br />Unpacking kubeadm (1.17.0-00) over (1.16.4-00) ...<br />Setting up kubeadm (1.17.0-00) ...<br />kubeadm set on hold.<br />k8@node01:~$ <br /><br />k8@node01:~$ sudo kubeadm upgrade node<br />[upgrade] Reading configuration from the cluster...<br />[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[upgrade] Skipping phase. Not a control plane node.<br />[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[upgrade] The configuration for this node was successfully updated!<br />[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.<br />k8@node01:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.17.0-00 kubectl=1.17.0-00 && sudo apt-mark hold kubelet kubectl<br />Canceled hold on kubelet.<br />Canceled hold on kubectl.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:4 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 166 not upgraded.<br />Need to get 27.9 MB of archives.<br />After this operation, 14.8 MB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.17.0-00 [8,742 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.17.0-00 [19.2 MB]<br />Fetched 27.9 MB in 4s (6,631 kB/s) <br />(Reading database ... 41117 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.17.0-00_amd64.deb ...<br />Unpacking kubectl (1.17.0-00) over (1.16.4-00) ...<br />Preparing to unpack .../kubelet_1.17.0-00_amd64.deb ...<br />Unpacking kubelet (1.17.0-00) over (1.16.4-00) ...<br />Setting up kubelet (1.17.0-00) ...<br />Setting up kubectl (1.17.0-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@node01:~$ sudo systemctl restart kubelet<br />k8@node01:~$ <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 216d v1.17.0 <br />node01 Ready,SchedulingDisabled <none> 215d v1.17.0 <= Kubernetes cluster upgraded<br />k8@master:~$ <br />k8@master:~$ kubectl uncordon node01<br />node/node01 uncordoned<br />k8@master:~$ <br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 216d v1.17.0<br />node01 Ready <none> 215d v1.17.0<br />k8@master:~$ <br />k8@master:~$ kubectl run nginx --image=nginx<br />kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.<br />deployment.apps/nginx created<br />k8@master:~$ kubectl get pods -o wide<br />NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES<br />nginx-6db489d4b7-lzqgd 1/1 Running 0 28s 192.168.177.242 node01 <none> <none><br />k8@master:~$ </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><br /></span></div>
<div class="gmail_default" style="color: black; font-family: "trebuchet ms" , sans-serif;">
<br /></div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-52156992278493188002020-01-04T18:32:00.001+05:302020-11-06T20:32:37.834+05:30Upgrading kubeadm clusters from v1.15 to v1.16 - Part 2<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace;"><span style="color: rgb(34 , 34 , 34);">upgrading minor version from v1.15.7 to one increment of latest patch version of 1.16.4. this would always be done in increments of one in minor versions. this shall be done first in the master node, followed later by worker node</span></span></div>
<span style="font-family: monospace;"><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.15.7<br />node01 Ready <none> 215d v1.15.7<br />k8@master:~$ <br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.7", GitCommit:"6c143d35bb11d74970e7bc0b6c45b6bfdffc0bd4", GitTreeState:"clean", BuildDate:"2019-12-11T12:40:15Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-get upgrade<br /><br />k8@master:~$ sudo apt-cache policy kubeadm<br />kubeadm:<br /> Installed: 1.14.2-00<br /> Candidate: 1.17.0-00<br /> Version table:<br /> 1.17.0-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.4-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages <== latest patch version<br /> 1.16.3-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.2-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.1-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.0-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.15.7-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages <br /> 1.15.6-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /><br />Marking master node to drain, <br /><br />k8@master:~$ kubectl cordon master<br />node/master cordoned<br />k8@master:~$<br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.15.7<br />node01 Ready <none> 215d v1.15.7<br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.16.4-00 && sudo apt-mark hold kubeadm<br />Canceled hold on kubeadm.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease <br />Hit:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:3 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubeadm<br />1 upgraded, 0 newly installed, 0 to remove and 225 not upgraded.<br />Need to get 8,767 kB of archives.<br />After this operation, 4,062 kB of additional disk space will be used.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.16.4-00 [8,767 kB]<br />Fetched 8,767 kB in 3s (3,145 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubeadm_1.16.4-00_amd64.deb ...<br />Unpacking kubeadm (1.16.4-00) over (1.15.7-00) ...<br />Setting up kubeadm (1.16.4-00) ...<br />kubeadm set on hold.<br />k8@master:~$ <br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.4", GitCommit:"224be7bdce5a9dd0c2fd0d46b83865648e2fe0ba", GitTreeState:"clean", BuildDate:"2019-12-11T12:44:45Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$ <br />k8@master:~$ sudo kubeadm upgrade plan <br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade] Fetching available versions to upgrade to<br />[upgrade/versions] Cluster version: v1.15.7<br />[upgrade/versions] kubeadm version: v1.16.4<br />I0104 12:37:04.235648 12589 version.go:251] remote version is much newer: v1.17.0; falling back to: stable-1.16<br />[upgrade/versions] Latest stable version: v1.16.4<br />[upgrade/versions] Latest version in the v1.15 series: v1.15.7<br /><br />Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':<br />COMPONENT CURRENT AVAILABLE<br />Kubelet 2 x v1.15.7 v1.16.4<br /><br />Upgrade to the latest stable version:<br /><br />COMPONENT CURRENT AVAILABLE<br />API Server v1.15.7 v1.16.4<br />Controller Manager v1.15.7 v1.16.4<br />Scheduler v1.15.7 v1.16.4<br />Kube Proxy v1.15.7 v1.16.4<br />CoreDNS 1.3.1 1.6.2<br />Etcd 3.3.10 3.3.15-0<br /><br />You can now apply the upgrade by executing the following command:<br /><br /> kubeadm upgrade apply v1.16.4<br /><br />_____________________________________________________________________<br /><br />k8@master:~$ sudo kubeadm upgrade apply v1.16.4<br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade/version] You have chosen to change the cluster version to "v1.16.4"<br />[upgrade/versions] Cluster version: v1.15.7<br />[upgrade/versions] kubeadm version: v1.16.4<br />[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y<br />[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]<br />[upgrade/prepull] Prepulling image for component etcd.<br />[upgrade/prepull] Prepulling image for component kube-controller-manager.<br />[upgrade/prepull] Prepulling image for component kube-apiserver.<br />[upgrade/prepull] Prepulling image for component kube-scheduler.<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[upgrade/prepull] Prepulled image for component kube-apiserver.<br />[upgrade/prepull] Prepulled image for component kube-controller-manager.<br />[upgrade/prepull] Prepulled image for component kube-scheduler.<br />[upgrade/prepull] Prepulled image for component etcd.<br />[upgrade/prepull] Successfully prepulled the images for all the control plane components<br />[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.16.4"...<br />Static pod: kube-apiserver-master hash: ec61aad55785ff79fecbf221a327876a<br />Static pod: kube-controller-manager-master hash: 496e9bb25dc11b4c7754d7492d434b2c<br />Static pod: kube-scheduler-master hash: 14ff2730e74c595cd255e47190f474fd<br />[upgrade/etcd] Upgrading to TLS for etcd<br />Static pod: etcd-master hash: 4ecda28ac93d555217d49e8a8885ac11<br />[upgrade/staticpods] Preparing for "etcd" upgrade<br />[upgrade/staticpods] Renewing etcd-server certificate<br />[upgrade/staticpods] Renewing etcd-peer certificate<br />[upgrade/staticpods] Renewing etcd-healthcheck-client certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/etcd.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-12-38-48/etcd.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: etcd-master hash: 4ecda28ac93d555217d49e8a8885ac11<br />Static pod: etcd-master hash: 4ecda28ac93d555217d49e8a8885ac11<br />Static pod: etcd-master hash: 210040bb1b64f944fc9ddbaad30e558c<br />[apiclient] Found 1 Pods for label selector component=etcd<br />[upgrade/staticpods] Component "etcd" upgraded successfully!<br />[upgrade/etcd] Waiting for etcd to become available<br />[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests142828755"<br />[upgrade/staticpods] Preparing for "kube-apiserver" upgrade<br />[upgrade/staticpods] Renewing apiserver certificate<br />[upgrade/staticpods] Renewing apiserver-kubelet-client certificate<br />[upgrade/staticpods] Renewing front-proxy-client certificate<br />[upgrade/staticpods] Renewing apiserver-etcd-client certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-12-38-48/kube-apiserver.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-apiserver-master hash: ec61aad55785ff79fecbf221a327876a<br />Static pod: kube-apiserver-master hash: ec61aad55785ff79fecbf221a327876a<br />Static pod: kube-apiserver-master hash: f2558d68a90916d30b1a3a116cf147f5<br />[apiclient] Found 1 Pods for label selector component=kube-apiserver<br />[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade<br />[upgrade/staticpods] Renewing controller-manager.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-12-38-48/kube-controller-manager.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-controller-manager-master hash: 496e9bb25dc11b4c7754d7492d434b2c<br />Static pod: kube-controller-manager-master hash: a72c7227785a50773e502c9b5e6f174e<br />[apiclient] Found 1 Pods for label selector component=kube-controller-manager<br />[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-scheduler" upgrade<br />[upgrade/staticpods] Renewing scheduler.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-12-38-48/kube-scheduler.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-scheduler-master hash: 14ff2730e74c595cd255e47190f474fd<br />Static pod: kube-scheduler-master hash: bbb6db8820f2306123bb7948fbf3411a<br />[apiclient] Found 1 Pods for label selector component=kube-scheduler<br />[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!<br />[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace<br />[kubelet] Creating a ConfigMap "kubelet-config-1.16" in namespace kube-system with the configuration for the kubelets in the cluster<br />[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials<br />[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token<br />[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster<br />[addons]: Migrating CoreDNS Corefile<br />[addons] Applied essential addon: CoreDNS<br />[addons] Applied essential addon: kube-proxy<br /><br />[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.16.4". Enjoy!<br /><br />[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.<br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.16.4-00 kubectl=1.16.4-00 && sudo apt-mark hold kubelet kubectl<br />Canceled hold on kubelet.<br />Canceled hold on kubectl.<br />Hit:1 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease <br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease <br />Hit:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease<br />Hit:3 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 224 not upgraded.<br />Need to get 30.0 MB of archives.<br />After this operation, 7,134 kB of additional disk space will be used.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.16.4-00 [9,233 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.16.4-00 [20.7 MB]<br />Fetched 30.0 MB in 7s (4,457 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.16.4-00_amd64.deb ...<br />Unpacking kubectl (1.16.4-00) over (1.15.7-00) ...<br />Preparing to unpack .../kubelet_1.16.4-00_amd64.deb ...<br />Unpacking kubelet (1.16.4-00) over (1.15.7-00) ...<br />Setting up kubelet (1.16.4-00) ...<br />Setting up kubectl (1.16.4-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@master:~$ <br /><br /><br />k8@master:~$ sudo systemctl restart kubelet<br />k8@master:~$ sudo systemctl status kubelet<br />● kubelet.service - kubelet: The Kubernetes Node Agent<br /> Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)<br /> Drop-In: /etc/systemd/system/kubelet.service.d<br /> └─10-kubeadm.conf<br /> Active: active (running) since Sat 2020-01-04 12:44:03 UTC; 14s ago<br /> Docs: <a href="https://kubernetes.io/docs/home/">https://kubernetes.io/docs/home/</a><br /> Main PID: 20798 (kubelet)<br /> Tasks: 19 (limit: 4565)<br /> Memory: 30.2M<br /> CGroup: /system.slice/kubelet.service<br /> └─20798 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --networ<br /><br /><br />k8@master:~$ kubectl uncordon master<br />node/master uncordoned<br />k8@master:~$ <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.16.4 <= master updated<br />node01 Ready <none> 215d v1.15.7<br />k8@master:~$ <br /><br /><br />+++++++++++++++= upgrading worker nodes +++++++++++++++<br /><br />k8@node01:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.16.4-00 && sudo apt-mark hold kubeadm<br />Canceled hold on kubeadm.<br />Hit:1 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease <br />Hit:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubeadm<br />1 upgraded, 0 newly installed, 0 to remove and 168 not upgraded.<br />Need to get 8,767 kB of archives.<br />After this operation, 4,062 kB of additional disk space will be used.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.16.4-00 [8,767 kB]<br />Fetched 8,767 kB in 2s (3,724 kB/s) <br />(Reading database ... 41117 files and directories currently installed.)<br />Preparing to unpack .../kubeadm_1.16.4-00_amd64.deb ...<br />Unpacking kubeadm (1.16.4-00) over (1.15.7-00) ...<br />Setting up kubeadm (1.16.4-00) ...<br />kubeadm set on hold.<br />k8@node01:~$ <br /><br />k8@master:~$ kubectl drain node01 --ignore-daemonsets<br />node/node01 cordoned<br />WARNING: ignoring DaemonSet-managed Pods: kube-system/calico-node-fdffm, kube-system/kube-proxy-zwv82, kube-system/weave-net-rlb5b<br />evicting pod "coredns-5644d7b6d9-shw2q"<br />evicting pod "nginx-7bb7cd8db5-mrzk4"<br />evicting pod "coredns-5644d7b6d9-bjzrc"<br />pod/coredns-5644d7b6d9-shw2q evicted<br />pod/coredns-5644d7b6d9-bjzrc evicted<br />pod/nginx-7bb7cd8db5-mrzk4 evicted<br />node/node01 evicted<br />k8@master:~$ <br /><br />k8@node01:~$ sudo kubeadm upgrade node<br />[upgrade] Reading configuration from the cluster...<br />[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[upgrade] Skipping phase. Not a control plane node[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[upgrade] The configuration for this node was successfully updated!<br />[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.<br />k8@node01:~$ <br /><br />k8@node01:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.4", GitCommit:"224be7bdce5a9dd0c2fd0d46b83865648e2fe0ba", GitTreeState:"clean", BuildDate:"2019-12-11T12:44:45Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}<br />k8@node01:~$</span><br />
<span style="font-family: monospace;"><br /></span>
<span style="font-family: monospace;">k8@node01</span><span style="font-family: monospace;">:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.16.4-00 kubectl=1.16.4-00 && sudo apt-mark hold kubelet kubectl</span><br />
<span style="font-family: monospace;">Canceled hold on kubelet.</span><br />
<span style="font-family: monospace;">Canceled hold on kubectl.</span><br />
<span style="font-family: monospace;">Hit:1 http://security.ubuntu.com/ubuntu cosmic-security InRelease</span><br />
<span style="font-family: monospace;">Hit:2 http://archive.ubuntu.com/ubuntu cosmic InRelease </span><br />
<span style="font-family: monospace;">Hit:4 http://archive.ubuntu.com/ubuntu cosmic-updates InRelease </span><br />
<span style="font-family: monospace;">Hit:5 http://archive.ubuntu.com/ubuntu cosmic-backports InRelease</span><br />
<span style="font-family: monospace;">Hit:3 https://packages.cloud.google.com/apt kubernetes-xenial InRelease </span><br />
<span style="font-family: monospace;">Reading package lists... Done</span><br />
<span style="font-family: monospace;">Reading package lists... Done</span><br />
<span style="font-family: monospace;">Building dependency tree </span><br />
<span style="font-family: monospace;">Reading state information... Done</span><br />
<span style="font-family: monospace;">The following packages will be upgraded:</span><br />
<span style="font-family: monospace;"> kubectl kubelet</span><br />
<span style="font-family: monospace;">2 upgraded, 0 newly installed, 0 to remove and 167 not upgraded.</span><br />
<span style="font-family: monospace;">Need to get 30.0 MB of archives.</span><br />
<span style="font-family: monospace;">After this operation, 7,134 kB of additional disk space will be used.</span><br />
<span style="font-family: monospace;">Get:1 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.16.4-00 [9,233 kB]</span><br />
<span style="font-family: monospace;">Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.16.4-00 [20.7 MB]</span><br />
<span style="font-family: monospace;">Fetched 30.0 MB in 9s (3,469 kB/s) </span><br />
<span style="font-family: monospace;">(Reading database ... 41117 files and directories currently installed.)</span><br />
<span style="font-family: monospace;">Preparing to unpack .../kubectl_1.16.4-00_amd64.deb ...</span><br />
<span style="font-family: monospace;">Unpacking kubectl (1.16.4-00) over (1.15.7-00) ...</span><br />
<span style="font-family: monospace;">Preparing to unpack .../kubelet_1.16.4-00_amd64.deb ...</span><br />
<span style="font-family: monospace;">Unpacking kubelet (1.16.4-00) over (1.15.7-00) ...</span><br />
<span style="font-family: monospace;">Setting up kubelet (1.16.4-00) ...</span><br />
<span style="font-family: monospace;">Setting up kubectl (1.16.4-00) ...</span><br />
<span style="font-family: monospace;">kubelet set on hold.</span><br />
<span style="font-family: monospace;">kubectl set on hold.</span><br />
<span style="font-family: monospace;">k8@node01</span><span style="font-family: monospace;">:~$ </span><br />
<div>
<br /></div>
<span style="font-family: monospace;">k8@node01</span><span style="font-family: monospace;">:~$ sudo systemctl restart kubelet</span><br />
<span style="font-family: monospace;">k8@node01</span><span style="font-family: monospace;">:~$ sudo systemctl status kubelet</span><br />
<span style="font-family: monospace;">● kubelet.service - kubelet: The Kubernetes Node Agent</span><br />
<span style="font-family: monospace;"> Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)</span><br />
<span style="font-family: monospace;"> Drop-In: /etc/systemd/system/kubelet.service.d</span><br />
<span style="font-family: monospace;"> └─10-kubeadm.conf</span><br />
<span style="font-family: monospace;"> Active: active (running) since Sun 2020-01-05 04:54:57 UTC; 5s ago</span><br />
<span style="font-family: monospace;"> Docs: https://kubernetes.io/docs/home/</span><br />
<span style="font-family: monospace;"> Main PID: 1467 (kubelet)</span><br />
<span style="font-family: monospace;"> Tasks: 9 (limit: 1134)</span><br />
<span style="font-family: monospace;"> Memory: 20.0M</span><br />
<span style="font-family: monospace;"> CGroup: /system.slice/kubelet.service</span><br />
<span style="font-family: monospace;"> └─1467 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --network</span><br />
<div>
<span style="font-family: monospace;">k8@node01</span></div>
<span style="font-family: monospace;"><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.16.4<br />node01 Ready,SchedulingDisabled <none> 215d v1.16.4<br />k8@master:~$ <br />k8@master:~$ kubectl uncordon node01<br />node/node01 uncordoned<br />k8@master:~$ <br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.16.4<br />node01 Ready <none> 215d v1.16.4<br />k8@master:~$</span><br />
<span style="font-family: monospace;"><br /></span>
<span style="font-family: monospace;">Try to see if you are able to deploy application.<br /><br />k8@master:~$ kubectl run nginx --image=nginx<br />kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.<br />deployment.apps/nginx created<br />k8@master:~$<br /><span class="gmail_default" style="color: rgb(0 , 0 , 0);"></span><br />k8@master:~$ kubectl get pods -o wide<br />NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES<br />nginx-6db489d4b7-7rskv 1/1 Running 0 9s 192.168.177.239 node01 <none> <none><br />k8@master:~$<span style="color: rgb(0 , 0 , 0);"></span></span><br />
<span style="font-family: monospace;"><br /></span></div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-54640375271021826672020-01-04T12:05:00.001+05:302020-11-06T20:32:44.930+05:30Upgrading kubeadm clusters from v1.14 to v1.15 - Part 1<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default" style="color: black;">
<span style="font-family: "comic sans ms" , sans-serif;">Trying to upgrade the kubenetes cluster, but we need to upgrade minor version to latest with increment of one<br />first, we shall upgrade master node and later worker nodes.<br /><br />current version:<br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.14.2<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br /><br />Drain master node for maintenance so that no new pods would be scheduled on the master node.<br /><br />k8@master:~$ kubectl cordon master<br />node/master cordoned<br />k8@master:~$ <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.14.2<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:20:34Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$ <br /><br />sudo apt-get upgrade<br />sudo apt-cache policy kubeadm<br /><br />you will now need to see the latest kubeadm with upgraded minor version .. I am looking at v1.15.0 <br /><br />k8@master:~$ sudo apt-cache policy kubeadm<br />kubeadm:<br /> Installed: 1.14.2-00<br /> Candidate: 1.17.0-00<br /> Version table:<br /> 1.17.0-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.4-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.3-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.2-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.1-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.0-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.15.7-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages <== last patch version<br /> 1.15.6-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /><br />upgrade control plane node, I would be using the latest patch version for v1.15.7 <br /><br />k8@master:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.15.7-00 && sudo apt-mark hold kubeadm<br />kubeadm was already not hold.<br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:3 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease <br />Hit:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following additional packages will be installed:<br /> cri-tools<br />The following packages will be upgraded:<br /> cri-tools kubeadm<br />2 upgraded, 0 newly installed, 0 to remove and 225 not upgraded.<br />Need to get 17.0 MB of archives.<br />After this operation, 1,663 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 cri-tools amd64 1.13.0-00 [8,776 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.15.7-00 [8,253 kB]<br />Fetched 17.0 MB in 8s (2,005 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../cri-tools_1.13.0-00_amd64.deb ...<br />Unpacking cri-tools (1.13.0-00) over (1.12.0-00) ...<br />Preparing to unpack .../kubeadm_1.15.7-00_amd64.deb ...<br />Unpacking kubeadm (1.15.7-00) over (1.14.2-00) ...<br />Setting up cri-tools (1.13.0-00) ...<br />Setting up kubeadm (1.15.7-00) ...<br />kubeadm set on hold.<br />k8@master:~$<br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.7", GitCommit:"6c143d35bb11d74970e7bc0b6c45b6bfdffc0bd4", GitTreeState:"clean", BuildDate:"2019-12-11T12:40:15Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$<br /><br />k8@master:~$ sudo kubeadm upgrade plan <br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade] Fetching available versions to upgrade to<br />[upgrade/versions] Cluster version: v1.14.2<br />[upgrade/versions] kubeadm version: v1.15.7<br />I0104 06:01:59.979160 29689 version.go:248] remote version is much newer: v1.17.0; falling back to: stable-1.15<br />[upgrade/versions] Latest stable version: v1.15.7<br />[upgrade/versions] Latest version in the v1.14 series: v1.14.10<br /><br />Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':<br />COMPONENT CURRENT AVAILABLE<br />Kubelet 2 x v1.14.2 v1.14.10<br /><br />Upgrade to the latest version in the v1.14 series:<br /><br />COMPONENT CURRENT AVAILABLE<br />API Server v1.14.2 v1.14.10<br />Controller Manager v1.14.2 v1.14.10<br />Scheduler v1.14.2 v1.14.10<br />Kube Proxy v1.14.2 v1.14.10<br />CoreDNS 1.3.1 1.3.1<br />Etcd 3.3.10 3.3.10<br /><br />You can now apply the upgrade by executing the following command:<br /><br /> kubeadm upgrade apply v1.14.10<br /><br />_____________________________________________________________________<br /><br />Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':<br />COMPONENT CURRENT AVAILABLE<br />Kubelet 2 x v1.14.2 v1.15.7<br /><br />Upgrade to the latest stable version:<br /><br />COMPONENT CURRENT AVAILABLE<br />API Server v1.14.2 v1.15.7<br />Controller Manager v1.14.2 v1.15.7<br />Scheduler v1.14.2 v1.15.7<br />Kube Proxy v1.14.2 v1.15.7<br />CoreDNS 1.3.1 1.3.1<br />Etcd 3.3.10 3.3.10<br /><br />You can now apply the upgrade by executing the following command:<br /><br /> kubeadm upgrade apply v1.15.7<br /><br />_____________________________________________________________________<br /><br /><br />k8@master:~$ sudo kubeadm upgrade apply v1.15.7<br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade/version] You have chosen to change the cluster version to "v1.15.7"<br />[upgrade/versions] Cluster version: v1.14.2<br />[upgrade/versions] kubeadm version: v1.15.7<br />[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y<br />[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]<br />[upgrade/prepull] Prepulling image for component etcd.<br />[upgrade/prepull] Prepulling image for component kube-apiserver.<br />[upgrade/prepull] Prepulling image for component kube-controller-manager.<br />[upgrade/prepull] Prepulling image for component kube-scheduler.<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[upgrade/prepull] Prepulled image for component etcd.<br />[upgrade/prepull] Prepulled image for component kube-apiserver.<br />[upgrade/prepull] Prepulled image for component kube-scheduler.<br />[upgrade/prepull] Prepulled image for component kube-controller-manager.<br />[upgrade/prepull] Successfully prepulled the images for all the control plane components<br />[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.15.7"...<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-scheduler-master hash: 9b290132363a92652555896288ca3f88<br />[upgrade/etcd] Upgrading to TLS for etcd<br />[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests124065540"<br />[upgrade/staticpods] Preparing for "kube-apiserver" upgrade<br />[upgrade/staticpods] Renewing apiserver certificate<br />[upgrade/staticpods] Renewing apiserver-kubelet-client certificate<br />[upgrade/staticpods] Renewing front-proxy-client certificate<br />[upgrade/staticpods] Renewing apiserver-etcd-client certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-06-03-39/kube-apiserver.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-apiserver-master hash: 461cf48224e9b4057addb8c3f5d64870<br />[apiclient] Found 1 Pods for label selector component=kube-apiserver<br />[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade<br />[upgrade/staticpods] Renewing controller-manager.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-06-03-39/kube-controller-manager.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: 9d89927ff1a0d70cf9452b3af5827f19<br />[apiclient] Found 1 Pods for label selector component=kube-controller-manager<br />[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-scheduler" upgrade<br />[upgrade/staticpods] Renewing scheduler.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-06-03-39/kube-scheduler.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-scheduler-master hash: 9b290132363a92652555896288ca3f88<br />Static pod: kube-scheduler-master hash: 7d6a1cec31a680b45724ee90bd535b49<br />[apiclient] Found 1 Pods for label selector component=kube-scheduler<br />[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!<br />[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace<br />[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster<br />[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials<br />[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token<br />[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster<br />[addons] Applied essential addon: CoreDNS<br />[addons] Applied essential addon: kube-proxy<br /><br />[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.15.7". Enjoy!<br /><br />[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.<br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.15.7-00 kubectl=1.15.7-00 && sudo apt-mark hold kubelet kubectl<br /><br />kubelet was already not hold.<br />kubectl was already not hold.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:4 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 224 not upgraded.<br />Need to get 29.0 MB of archives.<br />After this operation, 8,398 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.15.7-00 [8,760 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.15.7-00 [20.3 MB]<br />Fetched 29.0 MB in 10s (3,033 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.15.7-00_amd64.deb ...<br />Unpacking kubectl (1.15.7-00) over (1.14.2-00) ...<br />Preparing to unpack .../kubelet_1.15.7-00_amd64.deb ...<br />Unpacking kubelet (1.15.7-00) over (1.14.2-00) ...<br />Setting up kubelet (1.15.7-00) ...<br />Setting up kubectl (1.15.7-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@master:~$ <br /><br /><br />k8@master:~$ sudo systemctl restart kubelet<br />k8@master:~$ sudo systemctl status kubelet<br />● kubelet.service - kubelet: The Kubernetes Node Agent<br /> Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)<br /> Drop-In: /etc/systemd/system/kubelet.service.d<br /> └─10-kubeadm.conf<br /> Active: active (running) since Sat 2020-01-04 06:08:20 UTC; 9s ago<br /> Docs: <a href="https://kubernetes.io/docs/home/">https://kubernetes.io/docs/home/</a><br /> Main PID: 4890 (kubelet)<br /> Tasks: 19 (limit: 4565)<br /> Memory: 27.6M<br /> CGroup: /system.slice/kubelet.service<br /> └─4890 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --network<br /><br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.15.7<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br /><br />uncordon the master nodes so that any new pods would be scheduled.<br /><br />k8@master:~$ kubectl uncordon master<br />node/master uncordoned<br />k8@master:~$ <br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.15.7 <== cluster has been upgraded !<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br />k8@master:~$ <br /><br /><br />+++++++++++++++= upgrading worker nodes +++++++++++++++ <br /><br />update kubeadm on the worker nodes. <br /><br />k8@node01:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.15.7-00 && sudo apt-mark hold kubeadm<br />kubeadm was already not hold.<br />Get:1 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease [88.7 kB]<br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease <br />Get:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease [88.7 kB] <br />Get:5 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/main i386 Packages [197 kB] <br />Get:3 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease [8,993 B] <br />Get:6 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease [74.6 kB] <br />Get:7 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/main i386 Packages [332 kB] <br />Ign:8 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 Packages <br />Get:9 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/main amd64 Packages [210 kB] <br />Get:10 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/main Translation-en [84.1 kB] <br />Get:11 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/universe i386 Packages [498 kB] <br />Get:12 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/main amd64 Packages [345 kB] <br />Ign:8 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 Packages <br />Get:13 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/main Translation-en [144 kB] <br />Get:14 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/universe amd64 Packages [501 kB] <br />Get:15 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/universe amd64 Packages [697 kB] <br />Get:8 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 Packages [274 kB] <br />Get:16 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/universe Translation-en [144 kB] <br />Get:17 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/multiverse amd64 Packages [3,744 B] <br />Get:18 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/multiverse i386 Packages [3,904 B] <br />Get:19 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/universe i386 Packages [692 kB] <br />Get:20 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/universe Translation-en [195 kB] <br />Get:21 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/multiverse i386 Packages [3,904 B] <br />Get:22 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/multiverse amd64 Packages [3,744 B]<br />Get:23 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports/universe i386 Packages [3,992 B]<br />Get:24 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports/universe amd64 Packages [3,996 B]<br />Fetched 4,597 kB in 3s (1,349 kB/s) <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following additional packages will be installed:<br /> cri-tools<br />The following packages will be upgraded:<br /> cri-tools kubeadm<br />2 upgraded, 0 newly installed, 0 to remove and 168 not upgraded.<br />Need to get 17.0 MB of archives.<br />After this operation, 1,663 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 cri-tools amd64 1.13.0-00 [8,776 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.15.7-00 [8,253 kB]<br />Fetched 17.0 MB in 4s (4,646 kB/s) <br />(Reading database ... 41117 files and directories currently installed.)<br />Preparing to unpack .../cri-tools_1.13.0-00_amd64.deb ...<br />Unpacking cri-tools (1.13.0-00) over (1.12.0-00) ...<br />Preparing to unpack .../kubeadm_1.15.7-00_amd64.deb ...<br />Unpacking kubeadm (1.15.7-00) over (1.14.2-00) ...<br />Setting up cri-tools (1.13.0-00) ...<br />Setting up kubeadm (1.15.7-00) ...<br />kubeadm set on hold.<br />k8@node01:~$ <br /><br />Drain worker node so that any new pods won't be scheduled, already existing nodes would be evicted to other nodes. <br /><br /><br />k8@master:~$ kubectl drain node01 --ignore-daemonsets<br />node/node01 cordoned<br />k8@master:~$<br /><br /><br />k8@node01:~$ sudo kubeadm upgrade node<br />[upgrade] Reading configuration from the cluster...<br />[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[upgrade] Skipping phase. Not a control plane node[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[upgrade] The configuration for this node was successfully updated!<br />[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.<br />k8@node01:~$ <br /><br />k8@node01:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.15.7-00 kubectl=1.15.7-00 && sudo apt-mark hold kubelet kubectl<br /><br />kubelet was already not hold.<br />kubectl was already not hold.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:4 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 224 not upgraded.<br />Need to get 29.0 MB of archives.<br />After this operation, 8,398 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.15.7-00 [8,760 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.15.7-00 [20.3 MB]<br />Fetched 29.0 MB in 10s (3,033 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.15.7-00_amd64.deb ...<br />Unpacking kubectl (1.15.7-00) over (1.14.2-00) ...<br />Preparing to unpack .../kubelet_1.15.7-00_amd64.deb ...<br />Unpacking kubelet (1.15.7-00) over (1.14.2-00) ...<br />Setting up kubelet (1.15.7-00) ...<br />Setting up kubectl (1.15.7-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@node01:~$ <br /><br /><br />k8@node01:~$ sudo systemctl restart kubelet<br />k8@node01:~$ sudo systemctl status kubelet<br />● kubelet.service - kubelet: The Kubernetes Node Agent<br /> Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)<br /> Drop-In: /etc/systemd/system/kubelet.service.d<br /> └─10-kubeadm.conf<br /> Active: active (running) since Sat 2020-01-04 06:24:13 UTC; 4s ago<br /> Docs: <a href="https://kubernetes.io/docs/home/">https://kubernetes.io/docs/home/</a><br /> Main PID: 19127 (kubelet)<br /> Tasks: 14 (limit: 1134)<br /> Memory: 31.4M<br /> CGroup: /system.slice/kubelet.service<br /> └─19127 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --networ<br /><br />k8@node01:~$ <br /><br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.15.7<br />node01 Ready,SchedulingDisabled <none> 214d v1.15.7<br />k8@master:~$ <br /><br />k8@master:~$ kubectl uncordon node01<br />node/node01 uncordoned<br />k8@master:~$ <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.15.7<br />node01 Ready <none> 214d v1.15.7<br />k8@master:~$ <br /><br />Trying to upgrade the kubenetes cluster, but we need to upgrade minor version to latest with increment of one<br />first, we shall upgrade master node and later worker nodes.<br /><br />current version:<br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.14.2<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br /><br />Drain master node for maintenance so that no new pods would be scheduled on the master node.<br /><br />k8@master:~$ kubectl cordon master<br />node/master cordoned<br />k8@master:~$ <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.14.2<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:20:34Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$ <br /><br />sudo apt-get upgrade<br />sudo apt-cache policy kubeadm<br /><br />you will now need to see the latest kubeadm with upgraded minor version .. I am looking at v1.15.0 <br /><br />k8@master:~$ sudo apt-cache policy kubeadm<br />kubeadm:<br /> Installed: 1.14.2-00<br /> Candidate: 1.17.0-00<br /> Version table:<br /> 1.17.0-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.4-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.3-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.2-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.1-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.16.0-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /> 1.15.7-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages <== last patch version<br /> 1.15.6-00 500<br /> 500 <a href="http://apt.kubernetes.io/">http://apt.kubernetes.io</a> kubernetes-xenial/main amd64 Packages<br /><br />upgrade control plane node, I would be using the latest patch version for v1.15.7 <br /><br />k8@master:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.15.7-00 && sudo apt-mark hold kubeadm<br />kubeadm was already not hold.<br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:3 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease <br />Hit:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done <br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following additional packages will be installed:<br /> cri-tools<br />The following packages will be upgraded:<br /> cri-tools kubeadm<br />2 upgraded, 0 newly installed, 0 to remove and 225 not upgraded.<br />Need to get 17.0 MB of archives.<br />After this operation, 1,663 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 cri-tools amd64 1.13.0-00 [8,776 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.15.7-00 [8,253 kB]<br />Fetched 17.0 MB in 8s (2,005 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../cri-tools_1.13.0-00_amd64.deb ...<br />Unpacking cri-tools (1.13.0-00) over (1.12.0-00) ...<br />Preparing to unpack .../kubeadm_1.15.7-00_amd64.deb ...<br />Unpacking kubeadm (1.15.7-00) over (1.14.2-00) ...<br />Setting up cri-tools (1.13.0-00) ...<br />Setting up kubeadm (1.15.7-00) ...<br />kubeadm set on hold.<br />k8@master:~$<br /><br />k8@master:~$ kubeadm version<br />kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.7", GitCommit:"6c143d35bb11d74970e7bc0b6c45b6bfdffc0bd4", GitTreeState:"clean", BuildDate:"2019-12-11T12:40:15Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}<br />k8@master:~$<br /><br />k8@master:~$ sudo kubeadm upgrade plan <br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade] Fetching available versions to upgrade to<br />[upgrade/versions] Cluster version: v1.14.2<br />[upgrade/versions] kubeadm version: v1.15.7<br />I0104 06:01:59.979160 29689 version.go:248] remote version is much newer: v1.17.0; falling back to: stable-1.15<br />[upgrade/versions] Latest stable version: v1.15.7<br />[upgrade/versions] Latest version in the v1.14 series: v1.14.10<br /><br />Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':<br />COMPONENT CURRENT AVAILABLE<br />Kubelet 2 x v1.14.2 v1.14.10<br /><br />Upgrade to the latest version in the v1.14 series:<br /><br />COMPONENT CURRENT AVAILABLE<br />API Server v1.14.2 v1.14.10<br />Controller Manager v1.14.2 v1.14.10<br />Scheduler v1.14.2 v1.14.10<br />Kube Proxy v1.14.2 v1.14.10<br />CoreDNS 1.3.1 1.3.1<br />Etcd 3.3.10 3.3.10<br /><br />You can now apply the upgrade by executing the following command:<br /><br /> kubeadm upgrade apply v1.14.10<br /><br />_____________________________________________________________________<br /><br />Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':<br />COMPONENT CURRENT AVAILABLE<br />Kubelet 2 x v1.14.2 v1.15.7<br /><br />Upgrade to the latest stable version:<br /><br />COMPONENT CURRENT AVAILABLE<br />API Server v1.14.2 v1.15.7<br />Controller Manager v1.14.2 v1.15.7<br />Scheduler v1.14.2 v1.15.7<br />Kube Proxy v1.14.2 v1.15.7<br />CoreDNS 1.3.1 1.3.1<br />Etcd 3.3.10 3.3.10<br /><br />You can now apply the upgrade by executing the following command:<br /><br /> kubeadm upgrade apply v1.15.7<br /><br />_____________________________________________________________________<br /><br /><br />k8@master:~$ sudo kubeadm upgrade apply v1.15.7<br />[upgrade/config] Making sure the configuration is correct:<br />[upgrade/config] Reading configuration from the cluster...<br />[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[preflight] Running pre-flight checks.<br />[upgrade] Making sure the cluster is healthy:<br />[upgrade/version] You have chosen to change the cluster version to "v1.15.7"<br />[upgrade/versions] Cluster version: v1.14.2<br />[upgrade/versions] kubeadm version: v1.15.7<br />[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y<br />[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]<br />[upgrade/prepull] Prepulling image for component etcd.<br />[upgrade/prepull] Prepulling image for component kube-apiserver.<br />[upgrade/prepull] Prepulling image for component kube-controller-manager.<br />[upgrade/prepull] Prepulling image for component kube-scheduler.<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler<br />[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd<br />[upgrade/prepull] Prepulled image for component etcd.<br />[upgrade/prepull] Prepulled image for component kube-apiserver.<br />[upgrade/prepull] Prepulled image for component kube-scheduler.<br />[upgrade/prepull] Prepulled image for component kube-controller-manager.<br />[upgrade/prepull] Successfully prepulled the images for all the control plane components<br />[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.15.7"...<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-scheduler-master hash: 9b290132363a92652555896288ca3f88<br />[upgrade/etcd] Upgrading to TLS for etcd<br />[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests124065540"<br />[upgrade/staticpods] Preparing for "kube-apiserver" upgrade<br />[upgrade/staticpods] Renewing apiserver certificate<br />[upgrade/staticpods] Renewing apiserver-kubelet-client certificate<br />[upgrade/staticpods] Renewing front-proxy-client certificate<br />[upgrade/staticpods] Renewing apiserver-etcd-client certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-06-03-39/kube-apiserver.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-apiserver-master hash: abc9138b5fe7a4d853cb54b606ef2b35<br />Static pod: kube-apiserver-master hash: 461cf48224e9b4057addb8c3f5d64870<br />[apiclient] Found 1 Pods for label selector component=kube-apiserver<br />[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade<br />[upgrade/staticpods] Renewing controller-manager.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-06-03-39/kube-controller-manager.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: f55e807a968b84be4948aa51916af06f<br />Static pod: kube-controller-manager-master hash: 9d89927ff1a0d70cf9452b3af5827f19<br />[apiclient] Found 1 Pods for label selector component=kube-controller-manager<br />[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!<br />[upgrade/staticpods] Preparing for "kube-scheduler" upgrade<br />[upgrade/staticpods] Renewing scheduler.conf certificate<br />[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-01-04-06-03-39/kube-scheduler.yaml"<br />[upgrade/staticpods] Waiting for the kubelet to restart the component<br />[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)<br />Static pod: kube-scheduler-master hash: 9b290132363a92652555896288ca3f88<br />Static pod: kube-scheduler-master hash: 7d6a1cec31a680b45724ee90bd535b49<br />[apiclient] Found 1 Pods for label selector component=kube-scheduler<br />[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!<br />[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace<br />[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster<br />[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials<br />[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token<br />[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster<br />[addons] Applied essential addon: CoreDNS<br />[addons] Applied essential addon: kube-proxy<br /><br />[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.15.7". Enjoy!<br /><br />[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.<br />k8@master:~$ <br /><br />k8@master:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.15.7-00 kubectl=1.15.7-00 && sudo apt-mark hold kubelet kubectl<br /><br />kubelet was already not hold.<br />kubectl was already not hold.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:4 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 224 not upgraded.<br />Need to get 29.0 MB of archives.<br />After this operation, 8,398 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.15.7-00 [8,760 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.15.7-00 [20.3 MB]<br />Fetched 29.0 MB in 10s (3,033 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.15.7-00_amd64.deb ...<br />Unpacking kubectl (1.15.7-00) over (1.14.2-00) ...<br />Preparing to unpack .../kubelet_1.15.7-00_amd64.deb ...<br />Unpacking kubelet (1.15.7-00) over (1.14.2-00) ...<br />Setting up kubelet (1.15.7-00) ...<br />Setting up kubectl (1.15.7-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@master:~$ <br /><br /><br />k8@master:~$ sudo systemctl restart kubelet<br />k8@master:~$ sudo systemctl status kubelet<br />● kubelet.service - kubelet: The Kubernetes Node Agent<br /> Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)<br /> Drop-In: /etc/systemd/system/kubelet.service.d<br /> └─10-kubeadm.conf<br /> Active: active (running) since Sat 2020-01-04 06:08:20 UTC; 9s ago<br /> Docs: <a href="https://kubernetes.io/docs/home/">https://kubernetes.io/docs/home/</a><br /> Main PID: 4890 (kubelet)<br /> Tasks: 19 (limit: 4565)<br /> Memory: 27.6M<br /> CGroup: /system.slice/kubelet.service<br /> └─4890 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --network<br /><br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready,SchedulingDisabled master 215d v1.15.7<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br /><br />uncordon the master nodes so that any new pods would be scheduled.<br /><br />k8@master:~$ kubectl uncordon master<br />node/master uncordoned<br />k8@master:~$ <br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.15.7 <== cluster has been upgraded !<br />node01 Ready <none> 214d v1.14.2<br />k8@master:~$ <br />k8@master:~$ <br /><br /><br />+++++++++++++++= upgrading worker nodes +++++++++++++++ <br /><br />update kubeadm on the worker nodes. <br /><br />k8@node01:~$ sudo apt-mark unhold kubeadm && sudo apt-get update && sudo apt-get install -y kubeadm=1.15.7-00 && sudo apt-mark hold kubeadm<br />kubeadm was already not hold.<br />Get:1 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease [88.7 kB]<br />Hit:2 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease <br />Get:4 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease [88.7 kB] <br />Get:5 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/main i386 Packages [197 kB] <br />Get:3 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease [8,993 B] <br />Get:6 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease [74.6 kB] <br />Get:7 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/main i386 Packages [332 kB] <br />Ign:8 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 Packages <br />Get:9 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/main amd64 Packages [210 kB] <br />Get:10 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/main Translation-en [84.1 kB] <br />Get:11 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/universe i386 Packages [498 kB] <br />Get:12 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/main amd64 Packages [345 kB] <br />Ign:8 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 Packages <br />Get:13 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/main Translation-en [144 kB] <br />Get:14 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/universe amd64 Packages [501 kB] <br />Get:15 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/universe amd64 Packages [697 kB] <br />Get:8 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 Packages [274 kB] <br />Get:16 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/universe Translation-en [144 kB] <br />Get:17 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/multiverse amd64 Packages [3,744 B] <br />Get:18 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security/multiverse i386 Packages [3,904 B] <br />Get:19 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/universe i386 Packages [692 kB] <br />Get:20 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/universe Translation-en [195 kB] <br />Get:21 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/multiverse i386 Packages [3,904 B] <br />Get:22 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates/multiverse amd64 Packages [3,744 B]<br />Get:23 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports/universe i386 Packages [3,992 B]<br />Get:24 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports/universe amd64 Packages [3,996 B]<br />Fetched 4,597 kB in 3s (1,349 kB/s) <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following additional packages will be installed:<br /> cri-tools<br />The following packages will be upgraded:<br /> cri-tools kubeadm<br />2 upgraded, 0 newly installed, 0 to remove and 168 not upgraded.<br />Need to get 17.0 MB of archives.<br />After this operation, 1,663 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 cri-tools amd64 1.13.0-00 [8,776 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubeadm amd64 1.15.7-00 [8,253 kB]<br />Fetched 17.0 MB in 4s (4,646 kB/s) <br />(Reading database ... 41117 files and directories currently installed.)<br />Preparing to unpack .../cri-tools_1.13.0-00_amd64.deb ...<br />Unpacking cri-tools (1.13.0-00) over (1.12.0-00) ...<br />Preparing to unpack .../kubeadm_1.15.7-00_amd64.deb ...<br />Unpacking kubeadm (1.15.7-00) over (1.14.2-00) ...<br />Setting up cri-tools (1.13.0-00) ...<br />Setting up kubeadm (1.15.7-00) ...<br />kubeadm set on hold.<br />k8@node01:~$ <br /><br />Drain worker node so that any new pods won't be scheduled, already existing nodes would be evicted to other nodes. <br /><br />k8@master:~$ kubectl drain node01 --ignore-daemonsets<br />node/node01 cordoned<br />k8@master:~$<br /><br />k8@node01:~$ sudo kubeadm upgrade node<br />[upgrade] Reading configuration from the cluster...<br />[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'<br />[upgrade] Skipping phase. Not a control plane node[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace<br />[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"<br />[upgrade] The configuration for this node was successfully updated!<br />[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.<br />k8@node01:~$ <br /><br />k8@node01:~$ sudo apt-mark unhold kubelet kubectl && sudo apt-get update && sudo apt-get install -y kubelet=1.15.7-00 kubectl=1.15.7-00 && sudo apt-mark hold kubelet kubectl<br /><br />kubelet was already not hold.<br />kubectl was already not hold.<br />Hit:1 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic InRelease<br />Hit:2 <a href="http://security.ubuntu.com/ubuntu">http://security.ubuntu.com/ubuntu</a> cosmic-security InRelease<br />Hit:3 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-updates InRelease <br />Hit:5 <a href="http://archive.ubuntu.com/ubuntu">http://archive.ubuntu.com/ubuntu</a> cosmic-backports InRelease <br />Hit:4 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial InRelease <br />Reading package lists... Done<br />Reading package lists... Done<br />Building dependency tree <br />Reading state information... Done<br />The following packages will be upgraded:<br /> kubectl kubelet<br />2 upgraded, 0 newly installed, 0 to remove and 224 not upgraded.<br />Need to get 29.0 MB of archives.<br />After this operation, 8,398 kB disk space will be freed.<br />Get:1 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubectl amd64 1.15.7-00 [8,760 kB]<br />Get:2 <a href="https://packages.cloud.google.com/apt">https://packages.cloud.google.com/apt</a> kubernetes-xenial/main amd64 kubelet amd64 1.15.7-00 [20.3 MB]<br />Fetched 29.0 MB in 10s (3,033 kB/s) <br />(Reading database ... 113172 files and directories currently installed.)<br />Preparing to unpack .../kubectl_1.15.7-00_amd64.deb ...<br />Unpacking kubectl (1.15.7-00) over (1.14.2-00) ...<br />Preparing to unpack .../kubelet_1.15.7-00_amd64.deb ...<br />Unpacking kubelet (1.15.7-00) over (1.14.2-00) ...<br />Setting up kubelet (1.15.7-00) ...<br />Setting up kubectl (1.15.7-00) ...<br />kubelet set on hold.<br />kubectl set on hold.<br />k8@node01:~$ <br /><br />k8@node01:~$ sudo systemctl restart kubelet<br />k8@node01:~$ sudo systemctl status kubelet<br />● kubelet.service - kubelet: The Kubernetes Node Agent<br /> Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)<br /> Drop-In: /etc/systemd/system/kubelet.service.d<br /> └─10-kubeadm.conf<br /> Active: active (running) since Sat 2020-01-04 06:24:13 UTC; 4s ago<br /> Docs: <a href="https://kubernetes.io/docs/home/">https://kubernetes.io/docs/home/</a><br /> Main PID: 19127 (kubelet)<br /> Tasks: 14 (limit: 1134)<br /> Memory: 31.4M<br /> CGroup: /system.slice/kubelet.service<br /> └─19127 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --networ<br /><br />k8@node01:~$ <br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d v1.15.7<br />node01 Ready,SchedulingDisabled <none> 214d v1.15.7<br />k8@master:~$ <br /><br />k8@master:~$ kubectl uncordon node01<br />node/node01 uncordoned<br />k8@master:~$ <br /><br />finally, we have upgraded kubernetes cluster to <b>v1.15.7. </b><br /><br />k8@master:~$ kubectl get nodes<br />NAME STATUS ROLES AGE VERSION<br />master Ready master 215d <b>v1.15.7</b><br />node01 Ready <none> 214d <b>v1.15.7</b><br />k8@master:~$ </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: "comic sans ms" , sans-serif;"><br /></span></div>
<div class="gmail_default" style="color: black;">
k8@master:~$ kubectl get deployments<br />
NAME READY UP-TO-DATE AVAILABLE AGE<br />
nginx 1/1 1 1 63s<br />
k8@master:~$ kubectl get pods -o wide<br />
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES<br />
nginx-7bb7cd8db5-mrzk4 1/1 Running 0 69s 192.168.177.232 knode <none> <none><br />
k8@master:~$<br />
<br />
<br />
<br /></div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-46000293436340062482018-03-10T12:41:00.000+05:302020-11-06T20:34:11.697+05:30Configure Evolution(MUA) for root account on Linux<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace">Tried to configure 'sendmail' and 'dovecot' and tested using command line to send emails by creating couple of local user accounts. Later, for easy access configured 'evolution' mail client.</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="font-family:monospace,monospace">In brief, here are the changes made on the system while I configured system.</span><br></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace" size="4"><b>sendmail: </b></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"># yum install sendmail sendmail-cf</font></div><div class="gmail_default"><font face="monospace, monospace"><font color="#000000"># vi /etc/mail/<a href="http://sendmail.mc">sendmail.mc</a></font><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"><br></font></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><font face="monospace, monospace"><b><div class="gmail_default" style="color:rgb(0,0,0);display:inline"></div>From:</b><br>DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl<br><b><div class="gmail_default" style="color:rgb(0,0,0);display:inline"></div>To</b><br>dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl</font></blockquote><div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"></font></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace">save and quit(:wq)</font></div></div></blockquote><div><font face="monospace, monospace"><br></font></div><div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"># m4 /etc/mail/<a href="http://sendmail.mc">sendmail.mc</a> > /etc/mail/<a href="http://sendmail.cf">sendmail.cf</a><br></font></div></div><div class="gmail_default" style="color:rgb(0,0,0)"><font face="monospace, monospace"># vi /etc/mail/local-host-names<br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><a href="http://localhost.com">localhost.com</a></font></div><div class="gmail_default"><span style="color:rgb(0,0,0)"><font face="monospace, monospace"><br></font></span></div><div class="gmail_default"><font face="monospace, monospace"><span style="color:rgb(0,0,0)"># service sendmail restart</span><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;color:rgb(0,0,0)"># chkconfig sendmail on</div> <br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace" size="4"><b>Dovecot:</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"># yum install dovecot</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default"># vi /etc/dovecot/dovecot.conf</div></font></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">From:</div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">#protocols = imap pop3 lmtp</div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default"><br></div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">To:</div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">protocols = imap pop3</div></font></div></blockquote><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"># vi /etc/dovecot/conf.d/10-mail.conf</div></font></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">From</div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">#mail_location =</div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default"><br></div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">To</div></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><div class="gmail_default">mail_location = mbox:~/mail:INBOX=/var/mail/%u</div><div class="gmail_default"><br></div></font></div></blockquote><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"># service dovecot restart</div></font><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"># chkconfig dovecot on</div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline">Configure your evolution mail client: </div><br></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"> </div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"># yum -y install evolution</div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQCYTG12yupM6HOzjiaqPtEhU-FWXGx0Amn9oXGqx7OVOQM7NVO4n-L0gcq9lOrOXiEgaON9zO3DdgHnzzKgwrwn53Lbx1nxIdMbMf3FLmZix4aEeK9iQvMdQz7iSNzWPeP-eFp5FnLCI/s1600/image-731022.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQCYTG12yupM6HOzjiaqPtEhU-FWXGx0Amn9oXGqx7OVOQM7NVO4n-L0gcq9lOrOXiEgaON9zO3DdgHnzzKgwrwn53Lbx1nxIdMbMf3FLmZix4aEeK9iQvMdQz7iSNzWPeP-eFp5FnLCI/s320/image-731022.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6531210442038987794" /></a><br><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlSR-c1EuGg8HdKQe6vVkIrhAfe-uFNECtG1BGxXFdnuftJL0-8QU0FG6QRYEAehF1QavOjmWxOInhgkKoG0myh_W75x3QPYAwlq3Kqt5bxIlynerDctMnCk51R8Xn_KN8xJ4qEw6drv4/s1600/image-734120.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlSR-c1EuGg8HdKQe6vVkIrhAfe-uFNECtG1BGxXFdnuftJL0-8QU0FG6QRYEAehF1QavOjmWxOInhgkKoG0myh_W75x3QPYAwlq3Kqt5bxIlynerDctMnCk51R8Xn_KN8xJ4qEw6drv4/s320/image-734120.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6531210460048128706" /></a><br><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEHUIf6fpZSDWNJvuYujopPmFv7Op6Svzb5SswlljheYPzmhjdcneZ5zWLNCilAQqck9Dw0aTMtojWQMR9cAlTRcFjGwD7-V_g2Kq30IBoz6W8Qys9psubUvs2dihsllEach1dFp1d98o/s1600/image-736842.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEHUIf6fpZSDWNJvuYujopPmFv7Op6Svzb5SswlljheYPzmhjdcneZ5zWLNCilAQqck9Dw0aTMtojWQMR9cAlTRcFjGwD7-V_g2Kq30IBoz6W8Qys9psubUvs2dihsllEach1dFp1d98o/s320/image-736842.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6531210471140747874" /></a><br><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><br></div></font></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijNlZLPk2gOtY2Xg9csiU8Ykfjq3Oizsq6h9yLK59wTh2EgQ0Y9lSx8GySZRVIku66fzpYxAkIRuTxtlZJV2r70bmjW9DB742yMo92fVEDSruVuQGOBdPB8BGS2f0CaqvOMjwgL5bKOsc/s1600/image-738746.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijNlZLPk2gOtY2Xg9csiU8Ykfjq3Oizsq6h9yLK59wTh2EgQ0Y9lSx8GySZRVIku66fzpYxAkIRuTxtlZJV2r70bmjW9DB742yMo92fVEDSruVuQGOBdPB8BGS2f0CaqvOMjwgL5bKOsc/s320/image-738746.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6531210477447298146" /></a><br><br></div></font></div><div></div><div><font color="#000000" face="monospace, monospace"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><br></div></font></div><div><font color="#000000"><div class="gmail_default" style="color:rgb(0,0,0);display:inline"><font face="monospace, monospace"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdQ3jr08qf3w-otuCIu059jmoczvYu1_LLQkKHJNqtFAcMvhT4dejHNUkdZoHQTZg_Bla2ckImVO0uVrzWIm_kjsBipCMaQ-IANkjwN47gYfBGWZCvNik_D1acAbsfxUVf6OaUFSz8I1U/s1600/image-740807.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdQ3jr08qf3w-otuCIu059jmoczvYu1_LLQkKHJNqtFAcMvhT4dejHNUkdZoHQTZg_Bla2ckImVO0uVrzWIm_kjsBipCMaQ-IANkjwN47gYfBGWZCvNik_D1acAbsfxUVf6OaUFSz8I1U/s320/image-740807.png" border="0" alt="" id="BLOGGER_PHOTO_ID_6531210487438208354" /></a><br></font><br></div></font></div></div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-6408023850726495562018-02-14T20:05:00.001+05:302020-11-06T20:34:24.581+05:30upgrade 'git' to 2.x Redhat/CentOS<div dir="ltr"><div class="gmail_default"><div class="gmail_default"><font color="#000000" face="monospace, monospace">Tried to upgrade 'git' to latest as I had few applications which was not supported with older git which suggested me for 2.x. while I, tried to 'yum update git' I couldn't find 2.x being in Redhat/Oracle repository. Later, followed to install manually and these steps were documented below instructions. </font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><b>Overview: </b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">- Ensure your packages are installed(curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker)</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">- Remove installed 'git' from system</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">- Download the latest version from <a href="http://kernel.org">kernel.org</a> (I would download git-2.8.6)</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">- Install git.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">- Make sure you add in PATH of your profile.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">- Check your 'git' version. </font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><b>Steps:</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]# <b>yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker -y</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Loaded plugins: aliases, changelog, kabi, ovl, presto, refresh-packagekit, security, tmprepo, ulninfo, verify, versionlock</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Loading support for kernel ABI</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Setting up Install Process</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package libcurl-devel-7.19.7-53.el6_9.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package expat-devel-2.0.1-13.el6_8.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package gettext-devel-0.17-18.el6.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package openssl-devel-1.0.1e-57.0.1.el6.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package zlib-devel-1.2.3-29.el6.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package gcc-4.4.7-18.el6.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Package perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64 already installed and latest version</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Nothing to do</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]# </font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]# <b>yum remove git -y</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Setting up Remove Process</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Resolving Dependencies</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">--> Running transaction check</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">---> Package git.x86_64 0:1.7.1-9.el6_9 will be erased</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">--> Processing Dependency: git = 1.7.1-9.el6_9 for package: perl-Git-1.7.1-9.el6_9.noarch</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">--> Running transaction check</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">---> Package perl-Git.noarch 0:1.7.1-9.el6_9 will be erased</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">--> Finished Dependency Resolution</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Removed: git.x86_64 0:1.7.1-9.el6_9</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Dependency Removed: perl-Git.noarch 0:1.7.1-9.el6_9</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Complete!</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]#</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]# <b>cd /usr/src</b></font></div><div class="gmail_default"><font face="monospace, monospace"><span style="color:rgb(0,0,0)">[root@localhost src]# </span><b style="color:rgb(0,0,0)">wget <a href="https://www.kernel.org/pub/software/scm/git/git-2.8.6.tar.gz">https://www.kernel.org/pub/software/scm/git/git-2.8.6.tar.gz</a></b><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">--2018-02-13 20:00:25-- <a href="https://www.kernel.org/pub/software/scm/git/git-2.8.6.tar.gz">https://www.kernel.org/pub/software/scm/git/git-2.8.6.tar.gz</a></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Resolving www.kernel.org... 147.75.110.187, 2604:1380:3000:3500::3</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Connecting to <a href="http://www.kernel.org">www.kernel.org</a>|147.75.110.187|:443... connected.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">HTTP request sent, awaiting response... 200 OK</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Length: 5859600 (5.6M) [application/x-gzip]</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Saving to: "git-2.8.6.tar.gz"</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">100%[===================================================================================================================================>] 5,859,600 185K/s in 41s</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">2018-02-13 20:01:07 (140 KB/s) - "git-2.8.6.tar.gz" saved [5859600/5859600]</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost src]# <b>tar xzf git-2.8.6.tar.gz</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost src]# <b>cd git-2.8.6</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost git-2.8.6]# <b>make prefix=/usr/local/git all</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GIT_VERSION = 2.8.6</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">* new build flags</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC credential-store.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">* new link flags</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC abspath.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC advice.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC alias.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC alloc.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC archive.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC archive-tar.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC archive-zip.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC argv-array.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">* new prefix flags</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC attr.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">CC base85.o</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost git-2.8.6]# <b>make prefix=/usr/local/git install</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">SUBDIR perl</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">/usr/bin/perl Makefile.PL PREFIX='/usr/local/git' INSTALL_BASE='' --localedir='/usr/local/git/share/locale'</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">Writing perl.mak for Git</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-add--interactive</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-difftool</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-archimport</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-cvsexportcommit</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-cvsimport</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-cvsserver</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-relink</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-send-email</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">GEN git-svn</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">.</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">./check_bindir "z$bindir" "z$execdir" "$bindir/git-add"</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost git-2.8.6]# <b>echo "export PATH=$PATH:/usr/local/git/bin" </b>>>/etc/bashrc</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost git-2.8.6]# <b>source /etc/bashrc</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost git-2.8.6]#</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace"><br></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]# <b>git --version</b></font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">git version 2.8.6</font></div><div class="gmail_default"><font color="#000000" face="monospace, monospace">[root@localhost ~]#</font></div><div style="color:rgb(0,0,0)"><font face="monospace, monospace"><br></font></div><div style="color:rgb(0,0,0)"><font face="monospace, monospace">Thanks</font></div></div> </div> milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-37003910720743699232017-12-18T19:30:00.000+05:302017-12-18T20:18:43.904+05:30How to Configure Chef CentOS/Redhat<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">We shall see how to deploy and configure chef on the local machine..</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif; font-size: large;"><b>Chef Components</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Chef consist of a Chef server, one or more workstations, and a node where the chef-client is installed.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>Chef Server:</b> This is the central hub server that stores the cookbooks and recipes uploaded from workstations, which is then accessed by chef-client for configuration deployment.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b>Chef Workstations:</b> This where recipes, cookbooks, and other chef configuration details are created or edited. All these are then pushed to the Chef server from the workstation, where they will be available to deploy to chef-client nodes.</span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b>Chef Client:</b> This the target node where the configurations are deployed in which the chef-client is installed. A node can be any machine (physical, virtual, cloud, network device, etc..)</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Below are the prerequisites which I would leave for the reader to pre-configure..</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">DNS resolution should work between ChefServer, ChefWorkstation, ChefClient or else add in /etc/hosts. I am using CentOS 7 for this setup.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>Hostnames(Roles)</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">cen1.localhost.com (ChefServer)</span></span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com (ChefClient)</span><br />
<span style="font-family: Courier New, Courier, monospace;">fedora.localhost.com (ChefWorkstation)</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif; font-size: large;"><b>Chef Server:</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Download latest version of chef core.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># wget https://packages.chef.io/files/stable/chef-server/12.17.5/el/7/chef-server-core-12.17.5-1.el7.x86_64.rpm</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># rpm -ivh chef-server-core-12.17.5-1.el7.x86_64.rpm</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Once the installation is complete, you must reconfigure the chef server components to make up the server to work together</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b>#chef-server-ctl reconfigure</b></span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">check the status of the server components using the following command...</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b># chef-server-ctl status</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;">run: bookshelf: (pid 1167) 64163s; run: log: (pid 1163) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: nginx: (pid 10259) 63407s; run: log: (pid 1144) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: oc_bifrost: (pid 1170) 64163s; run: log: (pid 1166) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: oc_id: (pid 1168) 64163s; run: log: (pid 10087) 63510s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: opscode-erchef: (pid 6741) 63791s; run: log: (pid 1192) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: opscode-expander: (pid 1162) 64163s; run: log: (pid 1161) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: opscode-solr4: (pid 1169) 64163s; run: log: (pid 1165) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: postgresql: (pid 1160) 64163s; run: log: (pid 1159) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: rabbitmq: (pid 1146) 64163s; run: log: (pid 1145) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">run: redis_lb: (pid 7752) 63535s; run: log: (pid 1152) 64163s</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">We need to create an admin user. This user will have access to make changes to the infrastructure components in the organization we will be creating. Below command will generate the RSA private key automatically and should be saved to some location</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># chef-server-ctl user-create admin admin admin admin@localhost.com password -f /etc/chef/admin.pem</b></span><br />
<span style="font-family: Courier New, Courier, monospace;">ERROR: Error connecting to https://127.0.0.1/users/, retry 1/5</span><br />
<span style="font-family: Courier New, Courier, monospace;">ERROR: Error connecting to https://127.0.0.1/users/, retry 2/5</span><br />
<span style="font-family: Courier New, Courier, monospace;">ERROR: Error connecting to https://127.0.0.1/users/, retry 3/5</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">If you receive any error ERROR: Error connecting to https://127.0.0.1/users/, retry 1/5</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">which means there is another application which is already running on port 80 or 443. After, I had shutdown service it worked.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>#chef-server-ctl user-create admin admin admin admin@localhost.com admin1 --filename /etc/chef/admin.pem</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">where,</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"> chef-server-ctl user-create -h will help you to understand above command.</span><br />
<span style="font-family: Courier New, Courier, monospace;"><USERNAME FIRST_NAME [MIDDLE_NAME] LAST_NAME EMAIL PASSWORD></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Now, create an ORG name to hold the configurations.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>#chef-server-ctl org-create localhost "localhost, Chef Server" --association_user admin --filename /etc/chef/localhost-validator.pem</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">where,</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"> chef-server-ctl org-create -h will help you to understand above command.</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Make sure you have your firewall ports being opened for http, https.</span></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: large;"><b></b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: large;"><b>Chef Workstation:</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Download the latest version of chefdk</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># wget https://packages.chef.io/files/stable/chefdk/2.4.17/el/7/chefdk-2.4.17-1.el7.x86_64.rpm</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># rpm -ivh chefdk-2.4.17-1.el7.x86_64.rpm</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># chef verify</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Verification of component 'fauxhai' succeeded.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'kitchen-vagrant' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'openssl' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'delivery-cli' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'git' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'berkshelf' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'tk-policyfile-provisioner' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'opscode-pushy-client' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'test-kitchen' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'chefspec' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'knife-spork' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'inspec' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'chef-dk' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'chef-sugar' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'chef-client' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'chef-provisioning' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'generated-cookbooks-pass-chefspec' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Verification of component 'package installation' succeeded.</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Install the git and generate chef repository.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># yum install git -y</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># chef generate repo chef-repo</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># ls</b></span><br />
<span style="font-family: Courier New, Courier, monospace;">chefignore cookbooks data_bags environments LICENSE README.md roles</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">You can add this directory to version control.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># git config --global user.name "username"</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># git config --global user.email "xyz@domainname.com"</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Now, let's create a hidden directory called ".chef" under the chef-repo directory. This hidden directory will hold the RSA keys that we created on the Chef server.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"># vim .gitignore</span></span><br />
<span style="font-family: Courier New, Courier, monospace;">.chef</span><br />
<span style="font-family: Courier New, Courier, monospace;">.gitignore</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Commit all the existing changes</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># git add .</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># git commit -m "Initial Commit"</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">The RSA keys (.pem) generated when setting up the Chef Server will now need to be placed on the workstation. Place it under "~/chef-repo/.chef" directory.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># scp -pr root@cen01.localhost.com:/etc/chef/admin.pem chef-repo/.chef</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b># scp -pr root@cen01.localhost.com:/etc/chef/localhost-validator.pem chef-repo/.chef</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Knife is a command line interface for between a local chef-repo and the Chef server. To make the knife to work with your chef environment, we need to configure it by creating knife.rb in the "~/chef-repo/.chef/" directory.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Now, create and edit the knife.rb file</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"># cat knife.rb</span></span><br />
<span style="font-family: Courier New, Courier, monospace;">current_dir = File.dirname(__FILE__)</span><br />
<span style="font-family: Courier New, Courier, monospace;">log_level :info</span><br />
<span style="font-family: Courier New, Courier, monospace;">log_location STDOUT</span><br />
<span style="font-family: Courier New, Courier, monospace;">node_name "admin"</span><br />
<span style="font-family: Courier New, Courier, monospace;">client_key "#{current_dir}/admin.pem"</span><br />
<span style="font-family: Courier New, Courier, monospace;">validation_client_name "localhost-validator"</span><br />
<span style="font-family: Courier New, Courier, monospace;">validation_key "#{current_dir}/localhost-validator.pem"</span><br />
<span style="font-family: Courier New, Courier, monospace;">chef_server_url "https://cen01.localhost.com/organizations/localhost"</span><br />
<span style="font-family: Courier New, Courier, monospace;">syntax_check_cache_path "#{ENV['HOME']}/.chef/syntaxcache"</span><br />
<span style="font-family: Courier New, Courier, monospace;">cookbook_path ["#{current_dir}/../cookbooks"]</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>node_name:</b> This the username with permission to authenticate to the Chef server. Username should match with the user that we created on the Chef server.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>client_key:</b> The location of the file that contains user key that we copied over from the Chef server.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>validation_client_name:</b> This should be your organization's short name followed by -validator.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">validation_key: The location of the file that contains validation key that we copied over from the Chef server. This key is used when a chef-client is registered with the Chef server.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;">chef_server_url: The URL of the Chef server. It should begin with https://, followed by IP addressor FQDN of Chef server, organization name at the end just after /organizations/.</span><br />
<span style="font-family: Courier New, Courier, monospace;">{current_dir} represents ~/chef-repo/.chef/ directory, assuming that knife.rb file is in ~/chef-repo/.chef/. So you don't have to write the fully qualified path.</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">we must be fetching chef Server SSL certificate on our workstation</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b># knife ssl fetch</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;">WARNING: Certificates from cen01.localhost.com will be fetched and placed in your trusted_cert</span><br />
<span style="font-family: Courier New, Courier, monospace;">directory (/home/sunlnx/Documents/chef-repo/.chef/trusted_certs).</span><br />
<span style="font-family: Courier New, Courier, monospace;">Knife has no means to verify these are the correct certificates. You should</span><br />
<span style="font-family: Courier New, Courier, monospace;">verify the authenticity of these certificates after downloading.</span><br />
<span style="font-family: Courier New, Courier, monospace;">Adding certificate for cen01_localhost_com in /home/sunlnx/Documents/chef-repo/.chef/trusted_certs/cen01_localhost_com.crt</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>#knife client list</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;">localhost-validator</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif; font-size: large;"><b>Bootstrapping</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Bootstrapping a node is a process of installing chef-client on a target machine so that it can run as a chef-client node and communicate with the chef server</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;"><b>#knife bootstrap cen02.localhost.com -x root -P <yourpassword> --sudo</b></span></span><br />
<span style="font-family: Courier New, Courier, monospace;">Doing old-style registration with the validation key at /home/sunlnx/Documents/chef-repo/.chef/localhost-validator.pem...</span><br />
<span style="font-family: Courier New, Courier, monospace;">Delete your validation key in order to use your user credentials instead</span><br />
<span style="font-family: Courier New, Courier, monospace;">Connecting to cen02.localhost.com</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com -----> Installing Chef Omnibus (-v 13)</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com downloading https://omnitruck-direct.chef.io/chef/install.sh</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com to file /tmp/install.sh.4190/install.sh</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com trying wget...</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com el 7 x86_64</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com Getting information for chef stable 13 for el...</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com downloading https://omnitruck-direct.chef.io/stable/chef/metadata?v=13&p=el&pv=7&m=x86_64</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com to file /tmp/install.sh.4195/metadata.txt</span><br />
<span style="font-family: Courier New, Courier, monospace;">.</span><br />
<span style="font-family: Courier New, Courier, monospace;">.</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com Running handlers:</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com Running handlers complete</span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com Chef Client finished, 0/0 resources updated in 05 seconds</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="font-family: Trebuchet MS, sans-serif;">Once the bootstrapping is complete, list down the nodes using the following command.</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><b>#knife node list</b></span><br />
<span style="font-family: Courier New, Courier, monospace;">cen02.localhost.com</span><br />
<span style="font-family: Courier New, Courier, monospace;">#</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: Trebuchet MS, sans-serif;"><br /></span>
<span style="color: black; font-family: Trebuchet MS, sans-serif;"></span></span><br />
<span style="font-family: Courier New, Courier, monospace;">Thank you</span></div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-41457288672371266832017-11-12T10:03:00.001+05:302017-12-18T19:56:56.957+05:30Docker Swarm Basic Tutorial<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">I tried to provide at my best basic tutorial on Docker Swarm on how to configure/join workers to manager(master) nodes and run services. orchestrating service through Docker Swarm.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">After reading this blog, you would definitely come across basic idea on the container orchestration... </span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>What is container Orchestration ?</b></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0);"><span style="font-family: monospace, monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="font-family: monospace, monospace;"><span style="color: rgb(0 , 0 , 0);">Container Orchestration systems is where the next action is likely to be in the movement towards Building, Shipping, Running containers at scale. </span><span style="color: rgb(0 , 0 , 0);">The list of most popular software that currently provides a solution for this are Kubernetes, Docker Swarm and others.</span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>Why do we need container orchestration ?</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Imagine you might had to run hundred of containers, you can easily see if they are running in a distributed mode, ensuring your cluster is up and running etc </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Few features are:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- health checks on containers</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- Launch exact count of containers for an particular Docker image</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- Scaling number of containers up and down depending on the load</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- Perform rolling update on softwares across containers</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- ... more </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">I hope reader would be aware of Docker basic commands ..</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">The first step is to create a set of Docker machines that will act as nodes in our Docker Swarm. I am going to create 5 Docker Machines, where one of them will act as the Manager (Leader) and the other will be worker nodes.</span></div>
<div class="gmail_default">
<span style="font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">I will be using below Docker machines as hostnames with their roles ..</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">node1(192.168.0.28) <- Manager</span></div>
<div class="gmail_default">
</div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">node2(192.168.0.27) <- Worker</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">node3(192.168.0.26) <- Worker</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">node4(192.168.0.25) <- Worker</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">node5(192.168.0.24) <- Worker</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Creating Swarm Cluster</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Once your machines are setup, you can now proceed with setting up swarm. The first thing to do is to initilize swarm. Login to node1 and initilize swarm</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker swarm init --advertise-addr 192.168.0.28</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Swarm initialized: current node (wga1nmopjbir2ks92rxntj9dz) is now a manager.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">To add a worker to this swarm, run the following command:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-13fkd65lfhh2d1yb31fd59tuk <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">You will also notice that the output mentions the docker swarm join command to use in case you want another node to join as a worker. Keep in mind that you can have a node join as a worker or as a manager. At any point in time, there is only one LEADER and the other manager nodes will be as backup in case the current LEADER opts out.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker node ls</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">wga1nmopjbir2ks92rxntj9dz * node1 Ready Active <span style="white-space: pre;"> </span> Leader</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">You could use tokens to join either other nodes as a manager or worker. see below on how to get tokens ?</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Joining as Worker Node</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">To find out the join command for a worker, trigger below command </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker swarm join-token worker</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">To add a worker to this swarm, run the following command:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-13fkd65lfhh2d1yb31fd59tuk <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Joining as Manager Node</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker swarm join-token manager</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">To add a manager to this swarm, run the following command:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-a9j3npou6i5pb90fs9h92ez3u <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Adding worker nodes to Swarm</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">you need to SSH to node{2..5} and join them as workers.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node2# docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-13fkd65lfhh2d1yb31fd59tuk <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">This node joined a swarm as a worker.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node2#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node3# docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-13fkd65lfhh2d1yb31fd59tuk <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">This node joined a swarm as a worker.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node3#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node4# docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-13fkd65lfhh2d1yb31fd59tuk <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">This node joined a swarm as a worker.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node4#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node5# docker swarm join --token SWMTKN-1-3j0boiejfa0dhp82ar51fl1yvsscuwk3n1jppvzafvbv6mycyo-13fkd65lfhh2d1yb31fd59tuk <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">This node joined a swarm as a worker.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node5#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Login to manager node(node1) and check the status</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker node ls</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">wga1nmopjbir2ks92rxntj9dz * node1 Ready Active Leader</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">4wzc1nesi1hxa3ekq753w0koq node2 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">4nrsuwslw149mg4nwlipmxxlo node3 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">jbdizz7gvfdz03shtgu23h8kv node4 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">afpi9osujgcsqkdiqg4tnz6nb node5 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">you could also execute 'docker info' and check out swarm section.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Swarm: active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> NodeID: wga1nmopjbir2ks92rxntj9dz</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> <b>Is Manager: true</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> ClusterID: 7xpyowph2y1q2q12rhrkzzfei</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b> Managers: 1 ==> Swarm markered as 1 manager among total 5 active nodes</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Nodes: 5</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Orchestration:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Task History Retention Limit: 5</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Raft:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Snapshot Interval: 10000</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Number of Old Snapshots to Retain: 0</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> <b>Heartbeat Tick: 1</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Election Tick: 3</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Dispatcher:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Heartbeat Period: 5 seconds</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> CA Configuration:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Expiry Duration: 3 months</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Force Rotate: 0</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Autolock Managers: false</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Root Rotation In Progress: false</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> Node Address: 192.168.0.28</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b> Manager Addresses:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b> <a href="http://192.168.0.28:2377/">192.168.0.28:2377</a></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Testing Swarm orchestration</b></span></div>
<div class="gmail_default">
</div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">we have swarm up and running, its time to schedule our containers on it. we will not focus on the application and no need for us to worry where the appilcation is going to run. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">we will tell the manager to run the containers for us and it will take care of scheduling our container and send these commands to our nodes to distribute.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">we will run 'nginx' container and expose 'port 80'. we could specify number of instances to launch via 'replicas'. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">All would be runing from the 'node1' managerical node and we could use for administration. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service create --replicas 5 -p 80:80 --name web nginx</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">lwvjgxcxrg5wqw2fwrgzosnnw </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Since --detach=false was not specified, tasks will be created in the backg</span><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">round.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">In a future release, --detach=false will become the default.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">you could now see the status of the service being orchestrated to different nodes.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service ls </b>PORTS </span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">ID NAME MODE REPLICAS nginx:latest *:80->80/tcp</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">MAGE PORTS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">lwvjgxcxrg5w web replicated 8/8 nginx:latest *:80->80/tcp</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service ps web</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">l00c9u3kmfbn web.1 nginx:latest node1 Running Running 5 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">nzzi7z3wvikl web.2 nginx:latest node2 Running Running 5 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">91uizczo4dp3 web.3 nginx:latest node3 Running Running 5 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">nor39f9gu3j2 web.4 nginx:latest node4 Running Running 5 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">mtn3et6gu5on web.5 nginx:latest node5 Running Running 5 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Accessing service</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">you can acess the service by hitting any of the manager or worker node. It does not matter any particular node does not have any container scheduled on it.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">curl <a href="http://192.168.0.28/">http://192.168.0.28:80</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">curl <a href="http://192.168.0.27/">http://192.168.0.27:80</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">curl <a href="http://192.168.0.26/">http://192.168.0.26:80</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">curl <a href="http://192.168.0.25/">http://192.168.0.25:80</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">curl <a href="http://192.168.0.24/">http://192.168.0.24:80</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# curl <a href="http://192.168.0.28/">http://192.168.0.28:80</a></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><!DOCTYPE html></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><html></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><head></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><title>Welcome to nginx!</title></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><style></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> body {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> width: 35em;</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> margin: 0 auto;</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> font-family: Tahoma, Verdana, Arial, sans-serif;</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></style></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></head></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><body></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><h1>Welcome to nginx!</h1></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><p>If you see this page, the nginx web server is successfully installed and</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">working. Further configuration is required.</p></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><p>For online documentation and support please refer to</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><a href="<a href="http://nginx.org/">http://nginx.org/</a>"><a href="http://nginx.org/">nginx.org</a></a>.<br/></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Commercial support is available at</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><a href="<a href="http://nginx.com/">http://nginx.com/</a>"><a href="http://nginx.com/">nginx.com</a></a>.</p></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><p><em>Thank you for using nginx.</em></p></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></body></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></html></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Scaling up and Scaling down</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Currently we are running 5 containers and lets assume we require 10 containers, fire below to scale up. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service scale web=10</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service ps web</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">l00c9u3kmfbn web.1 nginx:latest node1 Running Running 18 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">nzzi7z3wvikl web.2 nginx:latest node2 Running Running 18 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">91uizczo4dp3 web.3 nginx:latest node3 Running Running 18 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">nor39f9gu3j2 web.4 nginx:latest node4 Running Running 18 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">mtn3et6gu5on web.5 nginx:latest node5 Running Running 18 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">t1reh7fnzfg9 web.6 nginx:latest node3 Running Running 10 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">qiyegknsz90w web.7 nginx:latest node1 Running Running 10 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">zqsafvln29ft web.8 nginx:latest node2 Running Running 10 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">mx6ezdng0n1i web.9 nginx:latest node4 Running Running 16 seconds ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">xrmugsonlbvl web.10 nginx:latest node5 Running Running 16 seconds ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1# </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">you can reduce with same command, it would reduce your containers on the nodes.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service scale web=5 </b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Draining node</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">whenever the node is ACTIVE, it will be always ready to accept tasks from Manager. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1#docker node ls</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">wga1nmopjbir2ks92rxntj9dz * node1 Ready Active Leader</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">4wzc1nesi1hxa3ekq753w0koq node2 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">4nrsuwslw149mg4nwlipmxxlo node3 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">jbdizz7gvfdz03shtgu23h8kv node4 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">afpi9osujgcsqkdiqg4tnz6nb node5 Ready Active</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">When the node is active, it can receive new tasks:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- during a service update to scale up</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- during a rolling update</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- when you set another node to Drain availability</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">- when a task fails on another active node</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">But sometimes, we have to bring the Node down for some maintenance reason. This meant by setting the Availability to Drain mode. Let us try that with one of our nodes.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">since there are 10 containers runing, each node is having 2 containers on it. lets made one of the node drain and see how the containers are being orchestrated without disrupting service.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker inspect node5</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><snip></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "Spec": {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "Labels": {},</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> <b>"Role": "worker",</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> <b>"Availability": "active" ==> status set to active</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><snip></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> },</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "Status": {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "State": "ready",</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "Addr": "192.168.0.24"</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><snip></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker node update --availability drain node5</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">node5</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service ps web</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">l00c9u3kmfbn web.1 nginx:latest node1 Running Running 30 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">nzzi7z3wvikl web.2 nginx:latest node2 Running Running 30 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">91uizczo4dp3 web.3 nginx:latest node3 Running Running 30 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">nor39f9gu3j2 web.4 nginx:latest node4 Running Running 30 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>e5pdetk2lghs web.5 nginx:latest node3 Running Running less than a second ago</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>mtn3et6gu5on \_ web.5 nginx:latest node5 Shutdown Shutdown 1 second ago</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">t1reh7fnzfg9 web.6 nginx:latest node3 Running Running 21 minutes </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">agoqiyegknsz90w web.7 nginx:latest node1 Running Running 21 minutes </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">agozqsafvln29ft web.8 nginx:latest node2 Running Running 21 minutes </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">agomx6ezdng0n1i web.9 nginx:latest node4 Running Running 11 minutes ago</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>m23nbrdvb635 web.10 nginx:latest node4 Running Running less than a second ago</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>xrmugsonlbvl \_ web.10 nginx:latest node5 Shutdown Shutdown less than a second ago</b></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">root@node1#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker inspect node5</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><snip></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><span style="white-space: pre;"> </span>"Spec": {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "Labels": {},</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> "Role": "worker",</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> <b> "Availability": "drain"</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><snip></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">You could now see the containers on the node5 has been re-scheduled on node3 and node4. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">you could now get on with maintenance on node5 and once they are up, you would require to make them into active state. On active state, containers will again get replicated.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b><br /></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Remove service</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">You could remove the service with 'rm' command.</span></div>
<div class="gmail_default">
<b style="color: black; font-family: monospace,monospace;">root@node1# docker service rm web</b></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace; font-size: medium;"><b>Rolling upgrade</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>root@node1# docker service update --image <imagename>:<version> web</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Thanks for re-sharing...</span></div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-86574689869343839442017-09-09T17:06:00.001+05:302017-09-10T07:29:27.814+05:30Configure Mail Server & Setting up Mail Client: Fedora Linux 25<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Here, I would let you know how to configure <b>mail server(MTA) and client(MUA)</b> using evolution which is by default.</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b>Environment:</b> Fedora 25</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">we will understand few components before we setup email configurations/settings</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b>MUA(Mail User Agent) or Mail Client: </b>Application used to write/send/read email messages.</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">e.g Evolution, KMail, Outlook etc.. text based mail clients like pine, mail ..etc </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b>MTA(Mail Transfer Agent):</b>Transferring email messages from one computer to another(intranet or Internet). We would be configuring <b>postfix </b>in this section</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b>MDA(Mail Delivery Agent): </b>It will receive emails from the MTA and delivers them to relevant mailbox MUA(e.g <b>Dovecot</b>). There are few of the popular MDA which would remove unwanted email messages or spam before they reach MUA Inbox.(e.g Procmail ..etc )</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>SMTP(Simple Mail Transfer Protocol): </b>communicates language that the MTA use to talk to each other and transfer message back and forth.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span>
<span style="color: blue; font-family: monospace , monospace;"><b>Architecture</b></span><br />
<div class="separator" style="clear: both; color: black; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; color: black; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg07zORDmmBubu4gCAJBbbvtj0Ckx1zIiFQag84OiJtyBGqmlKEiEDm2nccCt45Or2FGMF13RkEF-5DZu-Ewmh9TN7fmWfW4CIcjMJ0wLXFd8SRT3WmEPIJ6Xbz3_OLZR2RboyARTefgjI/s1600/Mail.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="247" data-original-width="269" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg07zORDmmBubu4gCAJBbbvtj0Ckx1zIiFQag84OiJtyBGqmlKEiEDm2nccCt45Or2FGMF13RkEF-5DZu-Ewmh9TN7fmWfW4CIcjMJ0wLXFd8SRT3WmEPIJ6Xbz3_OLZR2RboyARTefgjI/s1600/Mail.JPG" /></a></div>
<div style="color: black;">
<span style="font-family: monospace , monospace;"><b><span style="color: blue; font-family: monospace , monospace;"><br /></span></b></span>
<span style="font-family: monospace , monospace;"><b><span style="color: blue; font-family: monospace , monospace;">Configuring MTA</span></b></span></div>
</div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"><br /></span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Login as 'root' to perform below steps.</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b>Note: </b>SElinux was disabled.</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">- install postfix.</span></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;">#dnf install postfix -y</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">-Take a backup copy of the file and copy paste below contents and change according to your infra setup.(lines marked in red)</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;">#mv /etc/postfix/main.cf /etc/postfix/main.cf.original</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"># cat /etc/postfix/main.cf</span></b></div>
<div class="gmail_default">
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 1 compatibility_level = 2</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 2 queue_directory = /var/spool/postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 3 command_directory = /usr/sbin</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 4 daemon_directory = /usr/libexec/postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 5 data_directory = /var/lib/postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 6 mail_owner = postfix</span></div>
<div class="gmail_default">
<span style="color: red; font-family: monospace , monospace;"> <b> 7 myhostname = <a href="http://fedora.localhost.com/">fedora.localhost.com</a></b></span></div>
<div class="gmail_default">
<b><span style="color: red; font-family: monospace , monospace;"> 8 mydomain = <a href="http://localhost.com/">localhost.com</a></span></b></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 9 myorigin = $mydomain</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 10 inet_interfaces = $myhostname</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 11 inet_protocols = ipv4</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"><span style="color: black;"> 12 mydestination = $myhostname, localhost.$mydomain, localhost, </span>$mydomain</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 13 unknown_local_recipient_reject_code = 550</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"><span style="color: black;"> </span><span style="color: red;"> <b><span style="color: red;">14</span><span style="color: red;"> mynetworks = <span style="color: red;">192.168.122.0/24, 127.0.0.0/8, 10.0.0.0/24</span></span></b></span></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 15 alias_maps = hash:/etc/aliases</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 16 alias_database = hash:/etc/aliases</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"> 17 smtpd_banner = $myhostname ESMTP</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 18 debug_peer_level = 2</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 19 debugger_command =</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 20 PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 21 ddd $daemon_directory/$process_name $process_id & sleep 5</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 22 sendmail_path = /usr/sbin/sendmail.postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 23 newaliases_path = /usr/bin/newaliases.postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 24 mailq_path = /usr/bin/mailq.postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 25 setgid_group = postdrop</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 26 html_directory = no</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 27 manpage_directory = /usr/share/man</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 28 sample_directory = /usr/share/doc/postfix/samples</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 29 readme_directory = /usr/share/doc/postfix/README_FILES</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 30 meta_directory = /etc/postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 31 shlib_directory = /usr/lib64/postfix</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 32 message_size_limit = 10485760</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"> 33 mailbox_size_limit = 1073741824</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b> </b> 34 smtpd_sasl_type = dovecot</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"> 35 smtpd_sasl_path = private/auth</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"> 36 smtpd_sasl_auth_enable = yes</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"> 37 smtpd_sasl_security_options = noanonymous</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"> 38 smtpd_sasl_local_domain = $myhostname</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"><span style="color: black;"> 39 smtpd_recipient_restrictions = </span><span style="color: black;">permit_mynetworks,permit_auth_destination,permit_sasl_authenticated,reject</span></span></div>
<div style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
</div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">- start service persistent after reboot </span></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"># systemctl start postfix</span></b></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"># systemctl enable postfix</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">- If you had your firewall being running, add service 'smtp'. </span></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;">#firewall-cmd --add-service=smtp --permanent</span></b></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;">#firewall-cmd --reload</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<b><span style="color: blue; font-family: monospace , monospace;">Configuring MDA</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">- install dovecot</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">#dnf install dovecot -y</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">-Take a backup copy of the file and copy paste below contents and change according to your infra setup.(lines marked in red)</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b>#mv /etc/dovecot/dovecot.conf </b><b>/etc/dovecot/dovecot.conf.original</b></span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"><b style="color: black;">#mv </b><span style="color: black;"><b>/etc/dovecot/conf.d/10-auth.conf /etc/dovecot/conf.d/10-auth.conf.original</b></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>#mv /etc/dovecot/conf.d/10-mail.conf /etc/dovecot/conf.d/10-mail.conf.original</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>#mv /etc/dovecot/conf.d/10-master.conf /etc/dovecot/conf.d/10-master.conf.original</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>#mv /etc/dovecot/conf.d/10-ssl.conf /etc/dovecot/conf.d/10-ssl.conf.original</b></span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><b># cat </b><b>/etc/dovecot/dovecot.conf </b></span></div>
</div>
<div class="gmail_default">
<div>
<span style="color: black; font-family: monospace , monospace;"> 1 protocols = imap pop3 lmtp</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: black;"> 2 listen = *,::</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 3 dict {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 4 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 5 !include conf.d/*.conf</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 6 !include_try local.conf</span></span></div>
<div>
<span style="color: rgb(0 , 0 , 0);"><span style="font-family: monospace , monospace;">#</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><br /></span></div>
<div style="color: black;">
<b><span style="font-family: monospace , monospace;"># cat /etc/dovecot/conf.d/10-auth.conf</span></b></div>
<div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 1 disable_plaintext_auth = no</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: black;"> 2 auth_mechanisms = plain login</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 3 !include auth-system.conf.ext</span></span></div>
<div>
<span style="color: rgb(0 , 0 , 0);"><span style="font-family: monospace , monospace;">#</span></span></div>
<div>
<span style="color: rgb(0 , 0 , 0);"><span style="font-family: monospace , monospace;"><br /></span></span></div>
<div style="color: black;">
<span style="font-family: monospace , monospace;"><b># cat /etc/dovecot/conf.d/10-mail.conf</b></span></div>
</div>
</div>
<div class="gmail_default">
<div class="gmail_default">
<span style="font-family: monospace , monospace;"></span><br />
<div class="gmail_default" style="display: inline;">
<span style="font-family: monospace , monospace;"><span style="color: black;"><b> </b> </span><span style="color: black;">1 mail_location = maildir:~/Maildir</span></span><br />
<span style="font-family: monospace , monospace;"> </span><span style="font-family: monospace, monospace;">2 namespace inbox {</span></div>
</div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 3 inbox = yes</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 4 }</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 5 protocol !indexer-worker {</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 6 }</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"> 7 mbox_write_locks = fcntl</span><span style="font-family: monospace , monospace;"></span><br />
<span style="font-family: monospace , monospace; font-weight: bold;">#</span></div>
<div style="color: rgb(0 , 0 , 0); font-weight: bold;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div style="color: rgb(0 , 0 , 0); font-weight: bold;">
<span style="font-family: monospace , monospace;"># cat /etc/dovecot/conf.d/10-master.conf</span></div>
<div>
<div>
<span style="color: black; font-family: monospace , monospace;"><b> </b> 1 service imap-login {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 2 inet_listener imap {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 3 }</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 4 inet_listener imaps {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 5 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 6 }</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 7 service pop3-login {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 8 inet_listener pop3 {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 9 }</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 10 inet_listener pop3s {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 11 }</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 12 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 13 service lmtp {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 14 unix_listener lmtp {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 15 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 16 }</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 17 service imap {</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 18 }</span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 19 service pop3 {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 20 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 21 service auth {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 22 unix_listener auth-userdb {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 23 }</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 24 unix_listener /var/spool/postfix/private/auth {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 25 mode = 0666</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 26 user = postfix</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 27 group = postfix</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 28 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> <span style="color: black;"> 29 }</span></span></span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 30 service auth-worker {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 31 }</span></div>
<div>
<span style="font-family: monospace , monospace;"><span style="color: rgb(0 , 0 , 0);"> 32 service dict {</span></span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 33 unix_listener dict {</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 34 }</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"> 35 }</span></div>
<div style="color: black;">
<span style="font-family: monospace , monospace;"><b>#</b></span></div>
</div>
<div style="color: black;">
<b><span style="font-family: monospace , monospace;"><br /></span></b></div>
<div>
<span style="font-family: monospace , monospace;"><b style="color: black;"># cat </b><span style="color: black;"><b>/etc/dovecot/conf.d/10-ssl.conf</b></span></span></div>
<div>
<div>
<span style="color: black; font-family: monospace , monospace;"><b> </b> 1 ssl = required</span></div>
<div style="color: black;">
<span style="font-family: monospace , monospace;"> 2 ssl_cert = </etc/pki/dovecot/certs/dovecot.pem</span><span style="font-family: monospace , monospace;"></span><br />
<span style="font-family: monospace , monospace;"> 3 ssl_key = </etc/pki/dovecot/private/dovecot.pem</span></div>
<div style="color: black;">
<span style="font-family: monospace , monospace;"> 4 ssl_cipher_list = PROFILE=SYSTEM</span><span style="font-family: monospace , monospace;"></span><br />
<span style="font-family: monospace , monospace; font-weight: bold;">#</span></div>
<div style="color: black; font-weight: bold;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div style="color: black; font-weight: bold;">
<span style="font-family: monospace , monospace;"><span style="font-weight: normal;">- start service persistent after reboot </span></span></div>
</div>
<div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"># systemctl start dovecot</span></b></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"># systemctl enable dovecot</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">- If you had your firewall being running, add service 'smtp'. </span></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;">#firewall-cmd --add-service={pop3,imap} --permanent</span></b></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;">#firewall-cmd --reload</span></b></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"><br /></span></b></div>
<div class="gmail_default">
<b><span style="color: blue; font-family: monospace , monospace;">Configure MUA</span></b></div>
<div class="gmail_default" style="color: black;">
<b><span style="font-family: monospace , monospace;"><br /></span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Click on 'Evolution' and configure as per below ...</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Edit -> Preferences -> Add ->Next </span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0DqjR8sURJVvFbvOOcXmAo2BMAOzET4cx0-uq0N8wlvIlwx87uti4nAuTGP49ihOB2Jsl113gIH7dxG6inKO0X8J2-VMz9_HTN9sODo5NTcF-KDcolG1yV59TQk3Sfj9VQHbD0Et664c/s1600/image-714732.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6463741162992496418" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0DqjR8sURJVvFbvOOcXmAo2BMAOzET4cx0-uq0N8wlvIlwx87uti4nAuTGP49ihOB2Jsl113gIH7dxG6inKO0X8J2-VMz9_HTN9sODo5NTcF-KDcolG1yV59TQk3Sfj9VQHbD0Et664c/s320/image-714732.png" /></a></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCgeUtA191gJ_lWSAq2dY-AX54a9xLcOcEhxdD-UZaT4WXY7WnSPjLHUSsn0NnJdvQa-1g91sR8viHyvCIP8ht08L_77Z3z-2OM-nqouB2MCX2HI3NOXe70HATjbXS5LHzDRzDZeTUgS4/s1600/image-716924.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6463741169959782786" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCgeUtA191gJ_lWSAq2dY-AX54a9xLcOcEhxdD-UZaT4WXY7WnSPjLHUSsn0NnJdvQa-1g91sR8viHyvCIP8ht08L_77Z3z-2OM-nqouB2MCX2HI3NOXe70HATjbXS5LHzDRzDZeTUgS4/s320/image-716924.png" /></a></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgS3Wk9DaIIIdsVss8WSXYxlWUS6WGpTB_gq-s-UlnnrmVLHydrQIggd3cUWo1Kl6DjJp3DQHSZJBDmESFOY-yVfksLpUrl136G8ebkYydUfbrAjwtpAb5l8xyE7HnBS3RJSVfZk-vTPYc/s1600/image-718559.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6463741178577741378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgS3Wk9DaIIIdsVss8WSXYxlWUS6WGpTB_gq-s-UlnnrmVLHydrQIggd3cUWo1Kl6DjJp3DQHSZJBDmESFOY-yVfksLpUrl136G8ebkYydUfbrAjwtpAb5l8xyE7HnBS3RJSVfZk-vTPYc/s320/image-718559.png" /></a></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlp5XYzG4kAcXnL52aO9Mg4znzjBkenw67oHgy1LwDOJvOBlPMjXixmD_3BlG6sCsWyoWM-eF_bxIASTdK6SZZEuAI2cV2tlGkoYjqK0cyCN3-dEtAg1NPfc7mMWFFg2V778-IqTqPB_g/s1600/image-720073.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6463741185432695826" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlp5XYzG4kAcXnL52aO9Mg4znzjBkenw67oHgy1LwDOJvOBlPMjXixmD_3BlG6sCsWyoWM-eF_bxIASTdK6SZZEuAI2cV2tlGkoYjqK0cyCN3-dEtAg1NPfc7mMWFFg2V778-IqTqPB_g/s320/image-720073.png" /></a></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Leave rest to default and continue to check [OK] .</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<b><span style="color: blue; font-family: monospace , monospace;">Testing</span></b></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Compose email and read yourself :)</span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXLoe0NjZI8dd285yzEIJ3obQXID58qqQDgdTiQphR1Z5-CFY9h4ti10BHdiGgYjcQASSiC-Fo2hpV-Zvmuz8eFGyiXgNvZS1S0dc3v2nylKZl1u769n9f3t61NPjtYsfTb2wxuMvsPG0/s1600/image-721725.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6463741194741922642" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXLoe0NjZI8dd285yzEIJ3obQXID58qqQDgdTiQphR1Z5-CFY9h4ti10BHdiGgYjcQASSiC-Fo2hpV-Zvmuz8eFGyiXgNvZS1S0dc3v2nylKZl1u769n9f3t61NPjtYsfTb2wxuMvsPG0/s320/image-721725.png" /></a></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgV8Oy2H-yO0Z2l0YkIoChYVuJB11L-W8EwnjfWkGJ7Oae2yPSjgyLIZoscpBjwIttMR_ibdAEKVDNGD03cQWuzkqqjVYHlRma99DfIcqGAxM1Ve2Ldi8Axn3MN_R76-_CoKFXAsLTxgXE/s1600/image-723478.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6463741201764089362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgV8Oy2H-yO0Z2l0YkIoChYVuJB11L-W8EwnjfWkGJ7Oae2yPSjgyLIZoscpBjwIttMR_ibdAEKVDNGD03cQWuzkqqjVYHlRma99DfIcqGAxM1Ve2Ldi8Axn3MN_R76-_CoKFXAsLTxgXE/s320/image-723478.png" /></a></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default" style="color: black;">
<span style="font-family: monospace , monospace;">Thanks</span></div>
</div>
</div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-49932124547439370772017-08-26T20:38:00.000+05:302020-11-06T20:34:38.247+05:30Installation of Jenkins in Linux<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">In this blog, I would just summarize installation of jenkins in few steps</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">1. Jenkins is Java based application, hence install Java version 8</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">2. Tomcat is required to deploy Jenkins war file, hence install Apache Tomcat version 9</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">3. Download Jenkins war file</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">4. Deploy Jenkins war file using Tomcat </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">5. Install suggested plugins</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<b>Environment:</b> CentOS / Redhat<br />
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">- Install latest Java version</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">#<b>yum install java-1.8.0-openjdk</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">- Download Apache Tomcat </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>#wget <span style="font-size: 14px;">wget <a href="https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.0.M10/bin/apache-tomcat-9.0.0.M10.tar.gz">https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.0.M10/bin/apache-tomcat-9.0.0.M10.tar.gz</a></span></b></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">#<b>tar -xzvf </b></span><span style="font-size: 14px;"><b>apache-tomcat-9.0.0.M10.tar.gz</b></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">#<b>mv <span style="font-size: 14px;">apache-tomcat-9.0.0.M10 apache9</span></b></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;">- Provide username and password for Apache Tomcat</span></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">#</span><b><span style="font-size: 14px;">mv </span><span style="font-size: 14px;">apache9/conf/</span><span style="font-size: 14px;">tomcat-users.xml </span><span style="font-size: 14px;">apache9/conf/</span><span style="font-size: 14px;">tomcat-users.xml.original</span></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">#<b>vim </b></span><b><span style="font-size: 14px;">apache9/conf/</span><span style="font-size: 14px;">tomcat-users.xml</span></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><br /></span></span></div>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="font-size: 14px;"></span><br />
<div class="gmail_default" style="display: inline;">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"></span></span></div>
</blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><?xml version="1.0" encoding="UTF-8"?></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><!--</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> Licensed to the Apache Software Foundation (ASF) under one or more</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> contributor license agreements. See the NOTICE file distributed with</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> this work for additional information regarding copyright ownership.</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> The ASF licenses this file to You under the Apache License, Version 2.0</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> (the "License"); you may not use this file except in compliance with</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> the License. You may obtain a copy of the License at</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><br /></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> <a href="http://www.apache.org/licenses/LICENSE-2.0">http://www.apache.org/licenses/LICENSE-2.0</a></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><br /></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> Unless required by applicable law or agreed to in writing, software</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> distributed under the License is distributed on an "AS IS" BASIS,</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> See the License for the specific language governing permissions and</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> limitations under the License.</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">--></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><tomcat-users xmlns="<a href="http://tomcat.apache.org/xml">http://tomcat.apache.org/xml</a>"</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> xmlns:xsi="<a href="http://www.w3.org/2001/XMLSchema-instance">http://www.w3.org/2001/XMLSchema-instance</a>"</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> xsi:schemaLocation="<a href="http://tomcat.apache.org/xml">http://tomcat.apache.org/xml</a> tomcat-users.xsd"</span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"> version="1.0"></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><role rolename="manager-gui"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><role rolename="manager-script"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><role rolename="manager-jmx"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><role rolename="manager-status"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><role rolename="admin-gui"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><role rolename="admin-script"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><user username="</span></span><br />
<div class="gmail_default" style="display: inline;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">jenkins</span></span></div>
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">" password="</span></span><br />
<div class="gmail_default" style="display: inline;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">jenkins</span></span></div>
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">
" roles="manager-gui,manager-script,manager-jmx,manager-status,admin-gui,admin-script"/></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"></tomcat-users></span></span></blockquote>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204 , 204 , 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></blockquote>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>#cd apache9/bin</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><b>#./startup.sh</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">[user@localhost bin]$ ./startup.sh</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">Using CATALINA_BASE: /home/vagrant/tomcat9</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">Using CATALINA_HOME: /home/vagrant/tomcat9</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">Using CATALINA_TMPDIR: /home/vagrant/tomcat9/temp</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">Using JRE_HOME: /usr</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">Using CLASSPATH: /home/vagrant/tomcat9/bin/bootstrap.jar:/home/vagrant/tomcat9/bin/tomcat-juli.jar</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">Tomcat started.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">[user@localhost bin]$</span></div>
<div>
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
</div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">- Open your local browser and point your URL to <b><a href="http://localhost:8080/">http://localhost:8080</a></b> to display Apache Tomcat</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEqXE5r_RNbrr0r5CRZk8vHFroalYi-pLA3bVeBEX4qfH74EQ70bN1Km1x1eUreAQgyeskAGPo45uL7IJlIPbkTOzEqsFyiwbGTtbuIKlbkB4EIYGRGWWHm-9W7TkZpRi8XrL6D0tmvgA/s1600/image-757380.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6458600702991371234" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEqXE5r_RNbrr0r5CRZk8vHFroalYi-pLA3bVeBEX4qfH74EQ70bN1Km1x1eUreAQgyeskAGPo45uL7IJlIPbkTOzEqsFyiwbGTtbuIKlbkB4EIYGRGWWHm-9W7TkZpRi8XrL6D0tmvgA/s320/image-757380.png" /></a></span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;">In-order to deploy jenkins war file click on "<b>Manager App</b>" , provide credentials written in <span style="font-size: 14px;">tomcat-users.xml</span></span></div>
<div class="gmail_default">
</div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;">In deploy section write your context path and WAR file to be deloyed</span></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh35tqLdHbrup3x4ZDrFlv4hQ4jWCabTNA_eNuQGInkxNj-JpteWI1gERtuREN4QL2FP2cLTrko3k8uDgRkaPxNeOKKkQV8S1hr1UwphZW8dtF34C-pzZeIwF79lor_0-3KKCySmmH1dvk/s1600/image-759305.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6458600704364632690" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh35tqLdHbrup3x4ZDrFlv4hQ4jWCabTNA_eNuQGInkxNj-JpteWI1gERtuREN4QL2FP2cLTrko3k8uDgRkaPxNeOKKkQV8S1hr1UwphZW8dtF34C-pzZeIwF79lor_0-3KKCySmmH1dvk/s320/image-759305.png" /></a></span></span></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><br /></span></span></div>
<div class="gmail_default">
<span style="font-size: 14px;">Once above is completed, it will be deployed and in the Application and </span><b style="font-size: 14px;">click on Jenkins</b><span style="font-size: 14px;">, you need to unlock Jenkins by providing password located in your home directory. </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;">#</span></span><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace; font-size: 14px;"><b>cat /home/user/.jenkins/secret/</b></span><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace; font-size: 14px;"><b>initialAdminPassword</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><br /></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAeqIhYuoaycjP-XFR0kqwkz53sQcPBdpyasb7y7zI5CPXuVGD_9EbLNLHON6IVDPCPTGi9katGDbvSLFY9sPbLdwjELSWbvH7ODosQlJXkIkPV-K5W_ZdkMs0xePO2UGRZOfmveI12ds/s1600/image-760539.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6458600708355594338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAeqIhYuoaycjP-XFR0kqwkz53sQcPBdpyasb7y7zI5CPXuVGD_9EbLNLHON6IVDPCPTGi9katGDbvSLFY9sPbLdwjELSWbvH7ODosQlJXkIkPV-K5W_ZdkMs0xePO2UGRZOfmveI12ds/s320/image-760539.png" /></a></span></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace , monospace;"><span style="font-size: 14px;"><br /></span></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace; font-size: 14px;">Once they are unlocked, install the suggested plugins and get started. Create your First Admin user and Jenkins is ready </span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="font-size: 14px;"><span style="color: black; font-family: monospace , monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5Vm1uBaNdUfIWHgLufMLKJKqj0ABxBKM9qT68IIKyAiKUD9iRdbg4L_tWruPHZ52MT8bfSBJWvPGgVP91vUIyn0eFIThM_uHv53CnWcVqkvFJmYhR4Wck_cMjZ49yZL6XvQNlV4UUnFw/s1600/image-762458.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6458600717171522338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5Vm1uBaNdUfIWHgLufMLKJKqj0ABxBKM9qT68IIKyAiKUD9iRdbg4L_tWruPHZ52MT8bfSBJWvPGgVP91vUIyn0eFIThM_uHv53CnWcVqkvFJmYhR4Wck_cMjZ49yZL6XvQNlV4UUnFw/s320/image-762458.png" /></a></span></span><br />
<span style="font-size: 14px;"><br /></span></div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0tag:blogger.com,1999:blog-4652559897711604839.post-87039170425783156462017-06-10T07:23:00.001+05:302017-06-10T07:59:44.159+05:30Configure ELK stack for centralized log management<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="gmail_default">
<span style="font-family: monospace, monospace;">I had a two node centOS7 machine on which I had configured Elasticsearch, Logstash, Kibana(ELK) on the second node configured filebeat to send all logs to logstash. </span></div>
<div class="gmail_default">
<span style="font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="font-family: monospace, monospace;"><b>Hostnames : </b>node1 & node2</span></div>
<div class="gmail_default">
<span style="font-family: monospace, monospace;"><b>Environment :</b> CentOS 7.3</span></div>
<div class="gmail_default">
<span style="font-family: monospace , monospace;"><b>RPM versions :</b> elasticsearch/kibana/logstash/filebeat - 5.4.1</span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>Minimum requirement:</b> ensure your java package are installed and you have sufficient memory on the elasticserver based on the clients which you are trying to configure..</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Would install (elastic/logstash/kibana) on all in a single node(node1) and client(node2) would forward all the logs in /var/log/* to logstash on the node1.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Lets start ....</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>node1:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Download the RPM and install using yum, kibana is a tarball and shall extract to a directory.</span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">wget <a href="https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.1.rpm">https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.1.rpm</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">wget <a href="https://artifacts.elastic.co/downloads/logstash/logstash-5.4.1.rpm">https://artifacts.elastic.co/downloads/logstash/logstash-5.4.1.rpm</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">wget <a href="https://artifacts.elastic.co/downloads/kibana/kibana-5.4.1-linux-x86_64.tar.gz">https://artifacts.elastic.co/downloads/kibana/kibana-5.4.1-linux-x86_64.tar.gz</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<b style="color: black; font-family: monospace,monospace;">Install Elasticsearch:</b></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<b><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">[root@node1 ~]# systemctl daemon-reload</span></b></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 ~]# systemctl enable elasticsearch.service</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.</span></div>
<div class="gmail_default">
<b><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">[root@node1 ~]# systemctl start elasticsearch.service</span></b></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node1 ~]# systemctl status elasticsearch.service</b></span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>Install kibana:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 ~]# tar -xzvf kibana-5.4.1-linux-x86_64.tar.gz -C /usr/local</b></span></div>
<div class="gmail_default">
<b><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">[root@node1 ~]# cd /usr/local/</span></b></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 ~]# mv kiba* kibana</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Make it to start service when system boots</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 ~]# vim /etc/systemd/system/kibana.service</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[Service]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">ExecStart=/usr/local/kibana/bin/kibana</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[Install]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">WantedBy=multi-user.target</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<b><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">[root@node1 system]# systemctl enable kibana.service</span><span style="color: black; font-family: monospace, monospace;"><br /></span></b></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 system]# systemctl start kibana.service</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 system]# systemctl status kibana.service</b></span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Now you could point your ip:5601 on your browser to get your kibana dashboard.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>Install Logstash:</b></span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 ~]# cat /etc/logstash/conf.d/logstash.conf</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">input {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> beats {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> port => 5044</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> type => "logs"</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">filter {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> if [type] == "syslog" {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> grok {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}"}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> add_field => [ "received_at", "%{@timestamp}" ]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> add_field => [ "received_from", "%{hosts}" ]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> syslog_pri { }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> date {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">output {</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> elasticsearch {hosts => localhost}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> stdout { codec => rubydebug }</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[root@node1 ~]#</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node1 ~]# systemctl enable logstash</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Created symlink from /etc/systemd/system/multi-user.target.wants/logstash.service to /etc/systemd/system/logstash.service.</span></div>
<div class="gmail_default">
<b><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">[root@node1 ~]# systemctl start logstash.service</span></b></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node1 ~]# systemctl status logstash.service</b></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">node2:</span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b><br /></b></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>Install filebeat:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">Download and install filebeat RPM</span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">wget <a href="https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.4.1-x86_64.rpm">https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.4.1-x86_64.rpm</a></span></div>
<div class="gmail_default">
<br /></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node2 ~]# yum localinstall filebeat-5.4.1-x86_64.rpm</b></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node2 ~]# systemctl enable filebeat.service</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">Created symlink from /etc/systemd/system/multi-user.target.wants/filebeat.service to /usr/lib/systemd/system/filebeat.service.</span></div>
<div class="gmail_default">
<b><span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">[root@node2 ~]# systemctl start filebeat.service</span></b></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node2 ~]# systemctl status filebeat.service</b></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;">Make changes to your configuration logstash so that it would sent to your logstash server.</span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node2 ~]# vim /etc/filebeat/filebeat.yml</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> 91 output.logstash:</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> 92 # The Logstash hosts</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"> 93 hosts: ["<a href="http://192.168.122.100:5044/">192.168.122.100:5044</a>"]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>Error or Info:</b> Logs you need to check if something goes wrong </span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>node1:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>elasticsearch:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: rgb(0 , 0 , 0); font-family: monospace , monospace;"><b>[root@node1 ~]# cat /var/log/elasticsearch/elasticsearch.log</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></span><br />
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:34,760][INFO ][o.e.n.Node ] [] initializing ...</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:34,975][INFO ][o.e.e.NodeEnvironment ] [sfAWP7D] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [1.4gb], net total_space [6.1gb], spins? [unknown], types [rootfs]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:34,975][INFO ][o.e.e.NodeEnvironment ] [sfAWP7D] heap size [1.9gb], compressed ordinary object pointers [true]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:34,976][INFO ][o.e.n.Node ] node name [sfAWP7D] derived from node ID [sfAWP7DYQpiQTVtuRd0DFw]; set [<a href="http://node.name/">node.name</a>] to override</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:34,977][INFO ][o.e.n.Node ] version[5.4.1], pid[2966], build[2cfe0df/2017-05-29T16:05:51.443Z], OS[Linux/3.10.0-514.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_131/25.131-b12]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:34,977][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/share/elasticsearch]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,704][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [aggs-matrix-stats]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,716][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [ingest-common]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,716][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [lang-expression]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,716][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [lang-groovy]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,716][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [lang-mustache]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,716][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [lang-painless]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,717][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [percolator]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,717][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [reindex]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,717][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [transport-netty3]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,717][INFO ][o.e.p.PluginsService ] [sfAWP7D] loaded module [transport-netty4]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:37,717][INFO ][o.e.p.PluginsService ] [sfAWP7D] no plugins loaded</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:42,393][INFO ][o.e.d.DiscoveryModule ] [sfAWP7D] using discovery type [zen]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:43,857][INFO ][o.e.n.Node ] initialized</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:43,857][INFO ][o.e.n.Node ] [sfAWP7D] starting ...</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:44,190][INFO ][o.e.t.TransportService ] [sfAWP7D] publish_address {<a href="http://127.0.0.1:9300/">127.0.0.1:9300</a>}, bound_addresses {[::1]:9300}, {<a href="http://127.0.0.1:9300/">127.0.0.1:9300</a>}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T18:11:47,411][INFO ][o.e.c.s.ClusterService ] [sfAWP7D] new_master {sfAWP7D}{sfAWP7DYQpiQTVtuRd0DFw}{CNOzn0gIRBqP_pVHl-xusQ}{127.0.0.1}{<a href="http://127.0.0.1:9300/">127.0.0.1:9300</a>}, reason: zen-disco-elected-as-master ([0] nodes joined)</span></div>
<div>
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
</div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>logstash:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></span><br />
<div class="gmail_default" style="font-weight: bold;">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>[root@node1 ~]# cat /var/log/logstash/logstash-plain.log</b></span></div>
<div style="font-weight: bold;">
<span style="color: black; font-family: monospace, monospace;"><span style="font-weight: normal;">[2017-06-09T20:44:22,105][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[<a href="http://localhost:9200/]">http://localhost:9200/]</a>}}</span></span></div>
</div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"></span><br />
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:22,135][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=><a href="http://localhost:9200/">http://localhost:9200/</a>, :path=>"/"}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:22,395][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x554cf8d1 URL:<a href="http://localhost:9200/">http://localhost:9200/</a>>}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:22,422][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:22,567][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:22,599][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x74f64db6 URL://localhost>]}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:22,813][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:23,857][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"<a href="http://0.0.0.0:5044/">0.0.0.0:5044</a>"}</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:23,978][INFO ][logstash.pipeline ] Pipeline main started</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[2017-06-09T20:44:24,112][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}</span></div>
<div>
<span style="color: black; font-family: monospace, monospace;"><b><br /></b></span></div>
</div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>node2:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b><br /></b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><b>filebeat:</b></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">[root@node2 ~]#head -100 /var/log/filebeat/filebeat</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Setup Beat: filebeat; Version: 5.4.1</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Max Retries set to: 3</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Activated logstash as output plugin.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Publisher name: <a href="http://cen02.elktest.com/">cen02.elktest.com</a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Flush Interval set to: 1s</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Max Bulk Size set to: 2048</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO filebeat start running.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO No registry file found under: /var/lib/filebeat/registry. Creating a new registry file.</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Metrics logging every 30s</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Loading registrar data from /var/lib/filebeat/registry</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO States Loaded from registrar: 0</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Loading Prospectors: 1</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Prospector with previous states loaded: 0</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Starting prospector of type: log; id: 17005676086519951868</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Loading and starting Prospectors completed. Enabled prospectors: 1</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Starting Registrar</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Start sending events to output</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/wpa_supplicant.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/yum.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/VBoxGuestAdditions-uninstall.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/VBoxGuestAdditions.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/Xorg.0.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/boot.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;">2017-06-09T20:37:53+05:30 INFO Harvester started for file: /var/log/vboxadd-install.log</span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEe07II343frzCvngyDXJtz-CPt_T2MfeA1a_PQgMZ5TQ7daimIjZhyk9N4uxjse3BumbmCbFzZczPusoMecVU2JOfI9V6T4fvF02t9xAs7Hn1FmKWeHheXddIgN2y_C2ZC5svnWYovjc/s1600/image-706444.png"><img alt="" border="0" id="BLOGGER_PHOTO_ID_6429822052331511922" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEe07II343frzCvngyDXJtz-CPt_T2MfeA1a_PQgMZ5TQ7daimIjZhyk9N4uxjse3BumbmCbFzZczPusoMecVU2JOfI9V6T4fvF02t9xAs7Hn1FmKWeHheXddIgN2y_C2ZC5svnWYovjc/s320/image-706444.png" /></a></span></div>
<div class="gmail_default">
<span style="color: black; font-family: monospace, monospace;"><br /></span></div>
<div class="gmail_default">
<span style="font-family: monospace, monospace;">Thanks</span></div>
</div>
milearninghttp://www.blogger.com/profile/15276202890260201140noreply@blogger.com0