Tuesday, December 24, 2013

Getting the latest 2.7.x stable Puppet on Ubuntu and Redhat.

Quick guide for installing the latest 2.7.x stable version of Puppet.

I have a need to get a bunch of Ubuntu 10.x and 12.x LTS servers all running the same version of Puppet community.   I tried having a mix-mash of Puppet versions 3.x 2.x and 0.25.x; this does not work well, lots of random problems.

To solve this issue I am going to install the latest 2.x stable version, at the time of this writing (2013/12/24) it is 2.7.22.



Install the Puppetlabs apt-repository, puppet software, and hold the version:

Ubuntu 10.04:

$ wget http://apt.puppetlabs.com/puppetlabs-release-lucid.deb
$ sudo dpkg -i puppetlabs-release-lucid.deb
$ sudo apt-get update
$ sudo apt-get install puppet=2.7.22-1puppetlabs1 puppet-common=2.7.22-1puppetlabs1
$ echo "puppet hold" | sudo dpkg  --set-selections
$ echo "puppet-common hold" | sudo dpkg  --set-selections

Ubuntu 12.04:

$ wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb
$ sudo dpkg -i puppetlabs-release-precise.deb
$ sudo apt-get update
$ sudo apt-get install puppet=2.7.22-1puppetlabs1 puppet-common=2.7.22-1puppetlabs1
$ sudo apt-mark hold puppet puppet-common


NOTES: While the hold is set, no automatic security updates will be installed for Puppet.  Make sure you update as necessary.  



Additional Notes for Puppet server on Ubuntu:

12.04:

$ sudo apt-get install  puppet=2.7.22-1puppetlabs1 puppet-common=2.7.22-1puppetlabs1 puppetmaster=2.7.22-1puppetlabs1 puppetmaster-common=2.7.22-1puppetlabs1
$ sudo apt-mark hold  puppet puppet-common puppetmaster puppetmaster-common


Install Puppetlabs yum repository, puppet software, and exclude upgrades:

6:

$ sudo rpm -ivh http://yum-enterprise.puppetlabs.com/el/6/extras/x86_64/puppetlabs-enterprise-release-extras-6-2.noarch.rpm
$ sudo yum --showduplicates list puppet  # Put this here so I remember the command
$ sudo yum install puppet-2.7.22-1.el6
$ sudo vi /etc/yum.conf  # add exclude=puppet to [main] block


Monday, December 23, 2013

Fixing Rancid update problems (same diffs every day).

Rancid was email reports indicating the same changes were being made every day.  Basically new changes were not getting pushed into Rancid like they were supposed to.  

File permissions came to mind when I initially found the problem.   There is a FAQ that tells you how to resolve this.   We have had similar problem when someone ran Rancid as the 'root' user instead of 'rancid' like they should be.

This was not our problem, the problem was CVS was out of sync.


First check the status by running:

`cvs status`:

===================================================================
File: bn-001-0.nts.wustl.edu Status: Needs Checkout
   Working revision: 1.39 2013-12-22 22:43:15 -0600
   Repository revision: 1.39 /var/lib/rancid/CVS/wcdc/configs/bn-001-0.nts.wustl.edu,v
   Commit Identifier: boZWMtfaLtmxtsdx
   Sticky Tag: (none)
   Sticky Date: (none)
   Sticky Options: (none)
===================================================================
File: bn-002-0.nts.wustl.edu Status: Locally Modified
   Working revision: 1.18 2013-12-15 22:43:34 -0600
   Repository revision: 1.18 /var/lib/rancid/CVS/wcdc/configs/bn-002-0.nts.wustl.edu,v
   Commit Identifier: e6kEdcNzUdSp456x
   Sticky Tag: (none)
   Sticky Date: (none)
   Sticky Options: (none)


Run a update:

`cvs update`:

U bn-001-0.nts.wustl.edu
M bn-002-0.nts.wustl.edu


 Then run Rancid to see if this solved the problem:

`/var/lib/rancid/bin/rancid-run switches`


You will probably have to run it two times.   One to update and do a final check in.    One more run to make sure no more emails are generated.





Verify this fixed the problem:


`cvs status`:

===================================================================
File: bn-001-0.nts.wustl.edu    Status: Up-to-date
   Working revision:    1.39    2013-12-23 09:53:51 -0600
   Repository revision: 1.39    /var/lib/rancid/CVS/wcdc/configs/bn-001-0.nts.wustl.edu,v
   Commit Identifier:   boZWMtfaLtmxtsdx
   Sticky Tag:          (none)
   Sticky Date:         (none)
   Sticky Options:      -ko
===================================================================
File: bn-002-0.nts.wustl.edu    Status: Up-to-date
   Working revision:    1.19    2013-12-23 09:55:43 -0600
   Repository revision: 1.19    /var/lib/rancid/CVS/wcdc/configs/bn-002-0.nts.wustl.edu,v
   Commit Identifier:   maPVWYSoXBBu0hix
   Sticky Tag:          (none)
   Sticky Date:         (none)
   Sticky Options:      -ko




Monday, December 2, 2013

Using `strace` as a quick debugging tool.

The software SNAPP was throwing the error:

Failed to load dbi driver, please check db and/or dbi settings

In order to track down what was missing strace was used.  

By running the command with strace:

# strace snapp-collector -c /usr/local/etc/snapp_config.xml --foreground

The following was output:

execve("/usr/local/sbin/snapp-collector", ["snapp-collector", "-c", "/usr/local/etc/snapp_config.xml", "--foreground"], [/* 20 vars */]) = 0
brk(0)                                  = 0x1e5e000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2c9ca12000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=63119, ...}) = 0
mmap(NULL, 63119, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2c9ca02000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/libsnmp.so.15", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\27\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=620224, ...}) = 0
mmap(NULL, 2929152, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2c9c526000
mprotect(0x7f2c9c5bb000, 2093056, PROT_NONE) = 0
.......
mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fca66878000
mprotect(0x7fca66878000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fca668b7e30, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fca668b89d0, tls=0x7fca668b8700, child_tidptr=0x7fca668b89d0) = 58461
openat(AT_FDCWD, "/usr/lib/dbd", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
futex(0x7fca6c229a68, FUTEX_WAKE_PRIVATE, 2147483647) = 0
stat("/usr/local/etc/snapp_config.xml", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
stat("/usr/local/etc/snapp_config.xml", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
stat("/usr/local/etc/snapp_config.xml", {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
open("/usr/local/etc/snapp_config.xml", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=210, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fca66877000
read(3, "SNAPP"..., 16384) = 210
read(3, "", 12288)                      = 0
lseek(3, 0, SEEK_CUR)                   = 210
close(3)                                = 0
munmap(0x7fca66877000, 4096)            = 0
write(2, "Failed to load dbi driver, pleas"..., 63Failed to load dbi driver, please check db and/or dbi settings
) = 63
tgkill(58447, 58461, SIGRTMIN)          = 0
tgkill(58447, 58449, SIGRTMIN)          = 0
tgkill(58447, 58450, SIGRTMIN)          = 0
tgkill(58447, 58451, SIGRTMIN)          = 0
tgkill(58447, 58452, SIGRTMIN)          = 0
.....


To find the error look around the output for the error: "Failed to load dbi driver....", then look above that line until something looks like the obvious problem.   In this example, we know that dbi failed to load.   Looking up a few lines we find the error:

openat(AT_FDCWD, "/usr/lib/dbd", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Checking to see if this file exists:

ls -al /usr/lib/dbd
ls: cannot access /usr/lib/dbd: No such file or directory 

It does not.

Install the proper file:

apt-get install libdbd-mysql 

Run the program again and it works:

snapp-collector -c /usr/local/etc/snapp_config.xml --foreground
SNAPP-collector (version 3.0.7 (svn 9662)) started
collection_classes loaded=0, collections_loaded=0
config loaded
Snapp initialization complete 


Wednesday, November 27, 2013

Stop with the Proprietary Internet Applications.

It is 2013 and I still see this:






Why?   There is no technical reason for this.   There are so many open video technologies that work in ALL browsers.

Before you build a web site, post content, etc Google first and figure out how to do it properly.

I yearn for a day when the open standards, protocols, etc make the Internet a better place for every consumer.


Thursday, November 7, 2013

Using puppetlabs/firewall to manage iptables on Linux servers.

A while back, I rolled my own Puppet module to manage the iptables/ip6tables rules on all my servers at work.  This worked OK for a while, but now I am getting tired of debugging each little issue that pops up.

Puppetlabs has their own firewall management module now.


Here is my new configuration using their module.   Looking for a bit of feedback on this design before I pull the trigger, rip out mine, and roll with this new system.

First, install their module:
puppet module install puppetlabs/firewall

 I created my own module to support our unique environment:
cd ~
sudo puppet module generate duxklr-wufirewall
sudo mv duxklr-wufirewall /etc/puppet/modules/wufirewall


Initialize the firewall module...

/etc/puppet/modules/wufirewall/manifests/init.pp:
class wufirewall {
  resources { "firewall":
    purge => true
  }
  Firewall {
    before  => Class['wufirewall::postv4', 'wufirewall::postv6'],
    require => Class['wufirewall::prev4', 'wufirewall::prev6'],
  }
  class { ['wufirewall::prev4', 'wufirewall::postv4']: }
  class { ['wufirewall::prev6', 'wufirewall::postv6']: }
  class { 'firewall': }
}


The next set of modules, is the pre/post rules for ipv4/ipv6.   I had these mixed originally and used [] to specify ['iptables', 'ip6tables'], but that started failing on all the CentOS servers I was testing on.   Splitting ip4 and ip6 rules into different classes fixed the issue.   Need to look more into this, keeping them together seems cleaner.


/etc/puppet/modules/wufirewall/manifests/prev4.pp:
class wufirewall::prev4 {
  Firewall {
    require => undef,
  }
  # Default firewall rules
  firewall { '000 accept all icmp':
    proto   => 'icmp',
    action  => 'accept',
    provider => 'iptables',
  }
  firewall { '001 accept all to lo interface':
    proto   => 'all',
    iniface => 'lo',
    action  => 'accept',
    provider => 'iptables',
  }
  firewall { '002 accept related established rules':
    proto   => 'all',
    state   => ['RELATED', 'ESTABLISHED'],
    action  => 'accept',
    provider => 'iptables',
  }
  firewall { '010 allow ssh from Desktop Network':
    dport   => '22',
    proto  => 'tcp',
    source => '192.168.0.0/24',
    action => 'accept',
    provider => 'iptables',
  }
}

/etc/puppet/modules/wufirewall/manifests/prev6.pp:
class wufirewall::prev6 {
  Firewall {
    require => undef,
  }
  # Default firewall rules
  firewall { '000 ip6 accept all icmpv6':
    proto   => 'ipv6-icmp',
    action  => 'accept',
    provider => 'ip6tables',
  }
  firewall { '001 ip6 accept all to lo interface':
    proto   => 'all',
    iniface => 'lo',
    action  => 'accept',
    provider => 'ip6tables',
  }
  firewall { '002 ip6 accept related established rules':
    proto   => 'all',
    state   => ['RELATED', 'ESTABLISHED'],
    action  => 'accept',
    provider => 'ip6tables',
  }
  firewall { '011 ip6 allow sshv6 from desktop':
    dport   => '22',
    proto  => 'tcp',
    source => 'fe80::1/128',
    action => 'accept',
    provider => "ip6tables",
  }
}

/etc/puppet/modules/wufirewall/manifests/postv4.pp:
class wufirewall::postv4 {
  firewall { '900 log all drop connections':
    proto   => 'all',
    jump  => 'LOG',
    limit => '5/min',
    log_prefix => 'IPTables-Rejected: ',
    provider => 'iptables',
  }->
  firewall { '950 drop udp':
    proto   => 'udp',
    reject => 'icmp-port-unreachable',
    action  => 'reject',
    provider => 'iptables',
  }->
  firewall { '951 drop tcp':
    proto   => 'tcp',
    reject => 'tcp-reset',
    action  => 'reject',
    provider => 'iptables',
  }->
  firewall { '952 drop icmp':
    proto   => 'icmp',
    action  => 'drop',
    provider => 'iptables',
  }->
  firewall { '999 drop everything else - this is the failsafe rule':
    proto   => 'all',
    action  => 'drop',
    provider => 'iptables',
    before  => undef,
  }
}
/etc/puppet/modules/wufirewall/manifests/postv6.pp:
class wufirewall::postv6 {
  firewall { '900 ip6 log all drop connections':
    proto   => 'all',
    jump  => 'LOG',
    limit => '5/min',
    log_prefix => 'IPTables-Rejected: ',
    provider => 'ip6tables',
  }->
  firewall { '950 ip6 drop udpv6':
    proto   => 'udp',
    reject => 'icmp6-port-unreachable',
    action  => 'reject',
    provider => 'ip6tables',
  }->
  firewall { '951 ip6 drop tcp':
    proto   => 'tcp',
    reject => 'tcp-reset',
    action  => 'reject',
    provider => 'ip6tables',
  }->
  firewall { '952 ip6 drop icmp':
    proto   => 'icmp',
    action  => 'drop',
    provider => 'ip6tables',
  }->
  firewall { '999 ip6 drop everything else - this is the failsafe rule':
    proto   => 'all',
    action  => 'drop',
    provider => 'ip6tables',
    before  => undef,
  }
}

Here is where I am going to start putting all our one-off server rules.   We can create classes for common services like web, mail, dns, etc... But most of our systems run one off solutions that don't fit well into these categories.

/etc/puppet/modules/wufirewall/manifests/ubuntu_example_com.pp:
class wufirewall::ubuntu_example_com {
  firewall { '700 puppet on lab server':
    dport   => '8140',
    proto  => 'tcp',
    action => 'accept',
  }
}

Call the modules, here is the master site.pp...


/etc/puppet/manifests/site.pp:

## All servers get users managed by default
node 'default' {
  include manageusers
  include sudo
}
## Everything else should be explicitly set
node 'common' {
  include epel
  include wufirewall
  include sudo
  include manageusers
  include resolvconf
  include apt
  include munin
  include nrpe
  include rdiffbackup
  include syslog
  include openvmtools
}
node 'ubuntu.example.com' inherits 'common' {
  include wufirewall::ubuntu_example_com



So far most everything is working as expected:

jemurray@ubuntu:~$ sudo iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     icmp --  anywhere             anywhere             /* 000 accept all icmp */
ACCEPT     all  --  anywhere             anywhere             /* 001 accept all to lo interface */
ACCEPT     all  --  anywhere             anywhere             /* 002 accept related established rules */ state RELATED,ESTABLISHED
.......
LOG        all  --  anywhere             anywhere             /* 900 log all drop connections */ limit: avg 5/min burst 5 LOG level warning prefix "IPTables-Rejected: "
REJECT     udp  --  anywhere             anywhere             /* 950 drop udp */ reject-with icmp-port-unreachable
REJECT     tcp  --  anywhere             anywhere             /* 951 drop tcp */ reject-with tcp-reset
DROP       icmp --  anywhere             anywhere             /* 952 drop icmp */
DROP       all  --  anywhere             anywhere             /* 999 drop everything else - this is the failsafe rule */



One issue I am having is that when rules are removed from ip6tables, they are NOT actually removed from the running config.  Basically the 'purge' function is not working.   Still researching why.  Based on the forums other people are having the same problem.


This is my first run though this.   The rules need to be examined more thoroughly, more testing needs to be done, at this it is very beta.    I am more interested to validate I am on the right track....


Comments?

Thursday, October 17, 2013

Plotting Multiple Lines in GNUplot.

The end goal is to take a comma separated file and plot each value on a line graph using Gnuplot.  Nice X and Y axis labels and a color coded key will be included.   


The input file (wireless-devices2.csv) looks like this:

1377617462, 73, 251, 262, 633, 708, 1476, 454, 386, 466, 145, 608, 294, 895, 606, 336, 7612
1377617753, 69, 238, 263, 610, 692, 1392, 445, 378, 453, 135, 645, 280, 914, 613, 357, 7504
1377618054, 71, 218, 257, 601, 681, 1399, 439, 377, 398, 126, 630, 298, 933, 598, 363, 7408
1377618348, 60, 205, 259, 595, 686, 1412, 441, 363, 394, 131, 680, 299, 948, 601, 366, 7462
1377618649, 70, 209, 268, 595, 679, 1429, 419, 358, 394, 126, 662, 314, 946, 622, 376, 7485
1377618958, 76, 228, 268, 632, 683, 1420, 415, 369, 419, 128, 669, 314, 957, 643, 378, 7617
1377619259, 75, 232, 264, 663, 681, 1414, 422, 389, 439, 140, 724, 317, 962, 642, 374, 7758
1377619555, 80, 247, 268, 669, 687, 1397, 421, 405, 476, 140, 712, 342, 981, 645, 364, 7852
1377619854, 88, 255, 266, 671, 687, 1377, 425, 403, 491, 133, 749, 360, 1000, 658, 365, 7946
1377620161, 96, 253, 259, 666, 669, 1331, 416, 421, 522, 151, 777, 364, 996, 682, 378, 8000

The Gnuplot configuration (wd-total.gnuplot) file looks like this:

set datafile separator ","
set terminal png size 2000,600
set title "Wireless Users"
set ylabel "Wireless Users"
set ytics autofreq 500
set xlabel "Date"
set xdata time
set timefmt "%s"
set format x "%m/%d"
set key left top horizontal
set grid
plot "wireless-devices2.csv" using 1:2 with lines title 'Broken Device', \
  "wireless-devices2.csv" using 1:3 with lines title 'Meru Guest', \
  "wireless-devices2.csv" using 1:4 with lines title 'Meru WUFI', \
  "wireless-devices2.csv" using 1:5 with lines title 'Meru WUFI-S', \
  "wireless-devices2.csv" using 1:6 with lines title 'S40 wustl-guest-1.0', \
  "wireless-devices2.csv" using 1:7 with lines title 'S40 wustl-1.0', \
  "wireless-devices2.csv" using 1:8 with lines title 'S40 wustl-encrypted-1.0', \
  "wireless-devices2.csv" using 1:9 with lines title 'wustl-guest-1.0', \
  "wireless-devices2.csv" using 1:10 with lines title 'wustl-1.0', \
  "wireless-devices2.csv" using 1:11 with lines title 'wustl-encrypted-1.0', \
  "wireless-devices2.csv" using 1:12 with lines title 'Cisco WUFI-S', \
  "wireless-devices2.csv" using 1:13 with lines title 'Cisco WUFI', \
  "wireless-devices2.csv" using 1:14 with lines title 'wustl-2.0', \
  "wireless-devices2.csv" using 1:15 with lines title 'wustl-guest-2.0', \
  "wireless-devices2.csv" using 1:16 with lines title 'wustl-encrypted-2.0', \
  "wireless-devices2.csv" using 1:17 with lines lw 3 title 'TOTAL'


From the command line execute Gnuplot with the configuration file and image redirection as follows:


$ gnuplot < wd-total.gnuplot > wirelessdevices.png

The resulting file should look like this:








Wednesday, October 2, 2013

Apache + StartSSL + SSL (Free SSL for Everyone) or why every site should be SSL enabled.

Quick reference for getting a StartSSL certificate on Apache and Ubuntu server.

0) Review how to enable SSL in Apache first: https://help.ubuntu.com/lts/serverguide/httpd.html#https-configuration

1) Generate the SSL CSR (certificate signing request) with no password:
openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr

2a) Go to http://startssl.com

2b) Click Control Panel -> Validations Wizard -> Domain Validation (you must validate your domain first)

2c) Click Control Panel -> Certificate Wizard (skip automatic generation, paste the server.csr generated from the step 1, select your domain, enter additional domain names)

3) Copy the newly created server certificate from the website to a file called server.crt

4) Download the StartSSL CA bundel from: https://www.startssl.com/certs/sub.class1.server.ca.pem

5) Copy all the new files to the /etc/ssl directory:
cp server.key server.crt server.csr sub.class1.server.ca.pem /etc/ssl

6) Configure Apache with the following lines:
SSLCertificateFile "/etc/ssl/server.crt"
SSLCertificateKeyFile "/etc/ssl/server.key"
SSLCACertificateFile "/etc/ssl/sub.class1.server.ca.pem"

7) Restart Apache.

Use Puppet to manage iptables

I have made some updates to the puppet module to manage iptables on Ubuntu and RedHat.   Updates can be found here:

https://github.com/duxklr/iptables


There is still a lot more polishing that needs to be done.



Sunday, September 29, 2013

Septembers Technology Donation...

The GNU Project is 30 years old this month.  This project has contributed so much to the free software movement.

I look forward to a day when open software along with open hardware is a standard in every household.

In celebration of this 30 year mark, my monthly technology donation goes to the GNU project.


Keep moving forward my good friend!


Tuesday, July 30, 2013

How to identify Python variable data types?

Here is a quick way to tell what type of variable you are working with in Python by using the type() function:

jemurray@dsg:~ $ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information. 
>>> a = {}
>>> type(a)
>>> a = []
>>> type(a)
>>> a = ()
>>> type(a)
>>> a = ''
>>> type(a)

In many Python examples, the output is displayed something like this:

>>> print a
(1, 2, 3, 4)
If you didn't know what the () around the output meant, you can use the above example to figure it out.  


Tuesday, July 16, 2013

Puppet module to manage iptables?

I am looking for feedback on creating a module to manage iptables files on various Linux distributions with puppet.

I have a base design setup and working at: https://github.com/duxklr/iptables


The basics of the system so far is to:
  1. Use file templates to create a base iptables configuration that automatically gets pushed to all systems.   This will be contain the most basic rules that all servers should have.   
  2. Use an array to store additional host specific rules that will get processed by the template file.
    1. I selected a template/array style because our systems are all very one-off specific.   There is no good "grouping", service, module, etc way to categorize them.   

In the GIT repository above: 

You will find the .../templates/iptables-common-v4.erb which is the base file that all the iptables rules are built off of.  

In the .../manifests directory you will find a init.pp which is the heavy lifter in this case.   You will also find a file servers.pp.   This is the file that contains the host specific rules.   



The question I propose....  Is there a better way to manage the host specific rules?

I have looked into concat, but I don't see a elegant way to manage the very host specific rules that we have using this module.    If we have large amounts of servers that all preformed a similar function I can see this working very nicely.  

I have looked at all the other iptables modules in Puppet Forge, and none of them meet my individual needs.    




Thoughts?



Friday, July 12, 2013

Puppet error: Error 400 on SERVER: Could not find class

After building a puppet module, I was receiving the following error:


jemurray@ubuntu:~$ puppet agent --test
err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class copyFile for ubuntu.example.com at /etc/puppet/manifests/site.pp:1 on node ubuntu.example.com
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run


The problem was the module directory name.   You can't have upper case.   Change the module name 'copyFile' from upper case to lower case 'copyfile' and try again:

jemurray@ubuntu:~$ sudo puppet agent --test
[sudo] password for jemurray:
info: Caching catalog for ubuntu.example.com
info: Applying configuration version '1373689458'
notice: Finished catalog run in 0.03 seconds


Reference site here

Thursday, July 11, 2013

The July Technology for a Good Cause Donation

This is the first post of my monthly series, "Donations for a good technology cause".

For years, I have donated monthly to the OpenBSD foundation.   Specifically to fund the OpenSSH project.   SSH is is a tool I use every single day for work and play.   Probably one of the most useful pieces of software ever written.   I had a automatic monthly donation setup and everything was well.   Until, someone stole my credit card.   All my automatic payments got messed up.  When I tried to renew my OpenBSD subscription it didn't work.   At this point I figured other projects could use my help.   So now instead of only helping one organization, I am going to help a new project per month.  

This months donation goes to the Tor project.   I have been using Tor for many years.   I run a Tor exit node on the spare bandwidth from my Linode.

With all the media press about Internet privacy lately, I figured this was a good month to highlight this cause.  I think Bruce Schneier summed it up nicely, read the article (it is sort)...

If you are feeling generous don't a few bucks, or better yet run a Tor exit node.

Thursday, June 6, 2013

Performance profiling a Python script...

A Python script I wrote takes Infoblox DHCP logs from syslog and puts them info a Python sqlite database for historical storage and quick searches.

It was taking >55 minutes for the script to import a 250mb file.

What follows is an explanation of how I was able to significantly reduce this time.  


The first thing is try and get an understanding of what your script is doing and what pieces are taking the longest to execute.  To do this I setup a control group of 100,000 lines from the DHCP logs:

tail -100000 infoblox.log > infoblox-100000.log

Then I ran my script against this with the Python cProfiler:

python -m cProfile  ./lease2sql-b.py -o -f infoblox-100000.log 

This produces the following output:

         3012531 function calls (3012404 primitive calls) in 19.866 seconds
   Ordered by: standard name
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 UserDict.py:4(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:1077(Logger)
        1    0.000    0.000    0.000    0.000 __init__.py:1092(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:1104(setLevel)
   403646    0.178    0.000    0.542    0.000 __init__.py:1110(debug)
    30734    0.018    0.000    0.059    0.000 __init__.py:1122(info)
        1    0.000    0.000    0.000    0.000 __init__.py:1262(addHandler)
   434380    0.153    0.000    0.153    0.000 __init__.py:1310(getEffectiveLevel)
   434380    0.251    0.000    0.405    0.000 __init__.py:1324(isEnabledFor)
        1    0.000    0.000    0.000    0.000 __init__.py:1351(RootLogger)
        1    0.000    0.000    0.000    0.000 __init__.py:1357(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:1365(LoggerAdapter)
        1    0.000    0.000    0.000    0.000 __init__.py:1471(basicConfig)
    30734    0.049    0.000    0.113    0.000 __init__.py:1586(info)
   403646    0.383    0.000    0.961    0.000 __init__.py:1594(debug)
        1    0.000    0.000    0.000    0.000 __init__.py:1652(NullHandler)
        3    0.000    0.000    0.000    0.000 __init__.py:182(_checkLevel)
        3    0.000    0.000    0.000    0.000 __init__.py:210(_acquireLock)
        3    0.000    0.000    0.000    0.000 __init__.py:219(_releaseLock)
        1    0.000    0.000    0.000    0.000 __init__.py:230(LogRecord)
        2    0.002    0.001    0.006    0.003 __init__.py:24()
        1    0.000    0.000    0.000    0.000 __init__.py:346(Formatter)
        2    0.000    0.000    0.000    0.000 __init__.py:390(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:490(BufferingFormatter)
        1    0.000    0.000    0.000    0.000 __init__.py:532(Filter)
        1    0.000    0.000    0.000    0.000 __init__.py:569(Filterer)
        2    0.000    0.000    0.000    0.000 __init__.py:574(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:6()
        1    0.000    0.000    0.000    0.000 __init__.py:631(_addHandlerRef)
        1    0.000    0.000    0.000    0.000 __init__.py:641(Handler)
        1    0.000    0.000    0.000    0.000 __init__.py:650(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:679(createLock)
        1    0.000    0.000    0.000    0.000 __init__.py:749(setFormatter)
        1    0.000    0.000    0.000    0.000 __init__.py:805(StreamHandler)
        1    0.000    0.000    0.000    0.000 __init__.py:812(__init__)
        1    0.000    0.000    0.000    0.000 __init__.py:872(FileHandler)
        1    0.000    0.000    0.000    0.000 __init__.py:932(PlaceHolder)
        1    0.000    0.000    0.000    0.000 __init__.py:979(Manager)
        1    0.000    0.000    0.000    0.000 __init__.py:984(__init__)
        1    0.000    0.000    0.000    0.000 atexit.py:37(register)
        1    0.000    0.000    0.000    0.000 atexit.py:6()
        1    0.000    0.000    0.000    0.000 bisect.py:1()
        1    0.000    0.000    0.000    0.000 calendar.py:126(Calendar)
        1    0.000    0.000    0.000    0.000 calendar.py:132(__init__)
        1    0.000    0.000    0.000    0.000 calendar.py:138(setfirstweekday)
        1    0.000    0.000    0.000    0.000 calendar.py:21(IllegalMonthError)
        1    0.000    0.000    0.000    0.000 calendar.py:255(TextCalendar)
        1    0.000    0.000    0.000    0.000 calendar.py:28(IllegalWeekdayError)
        1    0.000    0.000    0.000    0.000 calendar.py:372(HTMLCalendar)
        1    0.000    0.000    0.000    0.000 calendar.py:47(_localized_month)
        1    0.000    0.000    0.000    0.000 calendar.py:484(TimeEncoding)
        1    0.000    0.000    0.000    0.000 calendar.py:496(LocaleTextCalendar)
        2    0.000    0.000    0.000    0.000 calendar.py:52(__init__)
        1    0.000    0.000    0.000    0.000 calendar.py:531(LocaleHTMLCalendar)
        1    0.000    0.000    0.000    0.000 calendar.py:6()
        1    0.000    0.000    0.000    0.000 calendar.py:66(_localized_day)
        2    0.000    0.000    0.000    0.000 calendar.py:71(__init__)
        1    0.001    0.001    0.002    0.002 collections.py:1()
        1    0.000    0.000    0.000    0.000 collections.py:25(OrderedDict)
        1    0.000    0.000    0.000    0.000 collections.py:355(Counter)
        1    0.002    0.002    0.002    0.002 dbapi2.py:24()
        1    0.000    0.000    0.000    0.000 dbapi2.py:55(register_adapters_and_converters)
        1    0.000    0.000    0.000    0.000 getopt.py:15()
        2    0.000    0.000    0.000    0.000 getopt.py:187(do_shorts)
        2    0.000    0.000    0.000    0.000 getopt.py:202(short_has_arg)
        1    0.000    0.000    0.000    0.000 getopt.py:38(GetoptError)
        1    0.000    0.000    0.000    0.000 getopt.py:51(getopt)
        1    0.001    0.001    0.001    0.001 heapq.py:31()
        1    0.000    0.000    0.000    0.000 keyword.py:11()
   100000    0.369    0.000   19.752    0.000 lease2sql-b.py:130(processline)
        1    0.090    0.090   19.842   19.842 lease2sql-b.py:174(onetimerun)
        1    0.000    0.000   19.842   19.842 lease2sql-b.py:218(main)
        1    0.007    0.007   19.866   19.866 lease2sql-b.py:55()
    15367    0.124    0.000   17.815    0.001 lease2sql-b.py:88(insertInfoDb)
        1    0.000    0.000    0.000    0.000 parser.py:147(_resultbase)
        1    0.000    0.000    0.000    0.000 parser.py:165(parserinfo)
        1    0.000    0.000    0.000    0.000 parser.py:200(__init__)
        7    0.000    0.000    0.000    0.000 parser.py:215(_convert)
        1    0.000    0.000    0.000    0.000 parser.py:290(parser)
        1    0.000    0.000    0.000    0.000 parser.py:292(__init__)
        1    0.000    0.000    0.000    0.000 parser.py:337(_result)
        1    0.000    0.000    0.000    0.000 parser.py:39(_timelex)
        1    0.007    0.007    0.010    0.010 parser.py:7()
        1    0.000    0.000    0.000    0.000 parser.py:700(_tzparser)
        1    0.000    0.000    0.000    0.000 parser.py:702(_result)
        1    0.000    0.000    0.000    0.000 parser.py:707(_attr)
        1    0.000    0.000    0.000    0.000 posixpath.py:43(normcase)
   100002    0.055    0.000    0.198    0.000 re.py:188(compile)
   100002    0.109    0.000    0.142    0.000 re.py:228(_compile)
        1    0.000    0.000    0.000    0.000 relativedelta.py:15(weekday)
        7    0.000    0.000    0.000    0.000 relativedelta.py:18(__init__)
        1    0.000    0.000    0.000    0.000 relativedelta.py:45(relativedelta)
        1    0.000    0.000    0.001    0.001 relativedelta.py:6()
       25    0.000    0.000    0.000    0.000 sre_compile.py:178(_compile_charset)
       25    0.000    0.000    0.000    0.000 sre_compile.py:207(_optimize_charset)
       20    0.000    0.000    0.000    0.000 sre_compile.py:24(_identityfunction)
     42/3    0.000    0.000    0.001    0.000 sre_compile.py:32(_compile)
       27    0.000    0.000    0.000    0.000 sre_compile.py:354(_simple)
        3    0.000    0.000    0.000    0.000 sre_compile.py:361(_compile_info)
        6    0.000    0.000    0.000    0.000 sre_compile.py:474(isstring)
        3    0.000    0.000    0.001    0.000 sre_compile.py:480(_code)
        3    0.000    0.000    0.003    0.001 sre_compile.py:495(compile)
      111    0.000    0.000    0.000    0.000 sre_parse.py:126(__len__)
      225    0.000    0.000    0.000    0.000 sre_parse.py:130(__getitem__)
       27    0.000    0.000    0.000    0.000 sre_parse.py:134(__setitem__)
      126    0.000    0.000    0.000    0.000 sre_parse.py:138(append)
    69/30    0.000    0.000    0.000    0.000 sre_parse.py:140(getwidth)
        3    0.000    0.000    0.000    0.000 sre_parse.py:178(__init__)
      251    0.000    0.000    0.000    0.000 sre_parse.py:182(__next)
      117    0.000    0.000    0.000    0.000 sre_parse.py:195(match)
      200    0.000    0.000    0.000    0.000 sre_parse.py:201(get)
       18    0.000    0.000    0.000    0.000 sre_parse.py:205(tell)
       33    0.000    0.000    0.000    0.000 sre_parse.py:257(_escape)
     13/3    0.000    0.000    0.001    0.000 sre_parse.py:301(_parse_sub)
     14/3    0.001    0.000    0.001    0.000 sre_parse.py:379(_parse)
        3    0.000    0.000    0.001    0.000 sre_parse.py:663(parse)
        3    0.000    0.000    0.000    0.000 sre_parse.py:67(__init__)
        9    0.000    0.000    0.000    0.000 sre_parse.py:72(opengroup)
        9    0.000    0.000    0.000    0.000 sre_parse.py:83(closegroup)
       42    0.000    0.000    0.000    0.000 sre_parse.py:90(__init__)
        1    0.000    0.000    0.003    0.003 threading.py:1()
        2    0.000    0.000    0.000    0.000 threading.py:101(RLock)
        1    0.000    0.000    0.000    0.000 threading.py:104(_RLock)
        2    0.000    0.000    0.000    0.000 threading.py:106(__init__)
        3    0.000    0.000    0.000    0.000 threading.py:121(acquire)
        3    0.000    0.000    0.000    0.000 threading.py:141(release)
        2    0.000    0.000    0.000    0.000 threading.py:181(Condition)
        1    0.000    0.000    0.000    0.000 threading.py:184(_Condition)
        2    0.000    0.000    0.000    0.000 threading.py:186(__init__)
        1    0.000    0.000    0.000    0.000 threading.py:226(_is_owned)
        1    0.000    0.000    0.000    0.000 threading.py:277(notify)
        1    0.000    0.000    0.000    0.000 threading.py:295(notifyAll)
        1    0.000    0.000    0.000    0.000 threading.py:304(_Semaphore)
        1    0.000    0.000    0.000    0.000 threading.py:352(_BoundedSemaphore)
        1    0.000    0.000    0.000    0.000 threading.py:364(Event)
        1    0.000    0.000    0.000    0.000 threading.py:367(_Event)
        1    0.000    0.000    0.000    0.000 threading.py:371(__init__)
        1    0.000    0.000    0.000    0.000 threading.py:385(set)
        1    0.000    0.000    0.000    0.000 threading.py:424(Thread)
        1    0.000    0.000    0.000    0.000 threading.py:436(__init__)
        1    0.000    0.000    0.000    0.000 threading.py:531(_set_ident)
        1    0.000    0.000    0.000    0.000 threading.py:56(_Verbose)
        6    0.000    0.000    0.000    0.000 threading.py:58(__init__)
        7    0.000    0.000    0.000    0.000 threading.py:63(_note)
        1    0.000    0.000    0.000    0.000 threading.py:733(_Timer)
        1    0.000    0.000    0.000    0.000 threading.py:762(_MainThread)
        1    0.000    0.000    0.000    0.000 threading.py:764(__init__)
        1    0.000    0.000    0.000    0.000 threading.py:771(_set_daemon)
        1    0.000    0.000    0.000    0.000 threading.py:802(_DummyThread)
        1    0.000    0.000    0.000    0.000 tz.py:152(_ttinfo)
        1    0.000    0.000    0.000    0.000 tz.py:191(tzfile)
        1    0.000    0.000    0.000    0.000 tz.py:31(tzutc)
        1    0.000    0.000    0.000    0.000 tz.py:475(tzrange)
        1    0.000    0.000    0.000    0.000 tz.py:54(tzoffset)
        1    0.000    0.000    0.000    0.000 tz.py:554(tzstr)
        1    0.001    0.001    0.002    0.002 tz.py:6()
        1    0.000    0.000    0.000    0.000 tz.py:629(_tzicalvtzcomp)
        1    0.000    0.000    0.000    0.000 tz.py:639(_tzicalvtz)
        1    0.000    0.000    0.000    0.000 tz.py:701(tzical)
        1    0.000    0.000    0.000    0.000 tz.py:83(tzlocal)
        1    0.000    0.000    0.000    0.000 tzwin.py:2()
        1    0.000    0.000    0.000    0.000 warnings.py:45(filterwarnings)
        1    0.000    0.000    0.000    0.000 weakref.py:47(__init__)
        1    0.000    0.000    0.000    0.000 {_sqlite3.connect}
        2    0.000    0.000    0.000    0.000 {_sqlite3.register_adapter}
        2    0.000    0.000    0.000    0.000 {_sqlite3.register_converter}
        3    0.000    0.000    0.000    0.000 {_sre.compile}
       12    0.000    0.000    0.000    0.000 {_sre.getlower}
        1    0.000    0.000    0.000    0.000 {built-in method utcfromtimestamp}
        3    0.000    0.000    0.000    0.000 {hasattr}
      287    0.000    0.000    0.000    0.000 {isinstance}
        1    0.000    0.000    0.000    0.000 {issubclass}
435198/435170    0.042    0.000    0.042    0.000 {len}
        2    0.000    0.000    0.000    0.000 {max}
        3    0.000    0.000    0.000    0.000 {method 'acquire' of 'thread.lock' objects}
      694    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
     7516    6.378    0.001    6.378    0.001 {method 'commit' of 'sqlite3.Connection' objects}
        1    0.000    0.000    0.000    0.000 {method 'cursor' of 'sqlite3.Connection' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    43541    9.485    0.000    9.485    0.000 {method 'execute' of 'sqlite3.Cursor' objects}
        2    0.000    0.000    0.000    0.000 {method 'extend' of 'list' objects}
    36025    1.715    0.000    1.715    0.000 {method 'fetchone' of 'sqlite3.Cursor' objects}
   100051    0.031    0.000    0.031    0.000 {method 'get' of 'dict' objects}
   134380    0.022    0.000    0.022    0.000 {method 'group' of '_sre.SRE_Match' objects}
        2    0.000    0.000    0.000    0.000 {method 'insert' of 'list' objects}
        3    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
       74    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
   100000    0.335    0.000    0.335    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
        2    0.000    0.000    0.000    0.000 {method 'release' of 'thread.lock' objects}
        9    0.000    0.000    0.000    0.000 {method 'remove' of 'list' objects}
   100000    0.052    0.000    0.052    0.000 {method 'rstrip' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {method 'setter' of 'property' objects}
        2    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
        6    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {method 'toordinal' of 'datetime.date' objects}
       86    0.000    0.000    0.000    0.000 {min}
        1    0.000    0.000    0.000    0.000 {open}
       96    0.000    0.000    0.000    0.000 {ord}
       17    0.000    0.000    0.000    0.000 {range}
        5    0.000    0.000    0.000    0.000 {thread.allocate_lock}
        8    0.000    0.000    0.000    0.000 {thread.get_ident}
        1    0.000    0.000    0.000    0.000 {time.localtime}
        1    0.000    0.000    0.000    0.000 {time.time}


It took almost 20 seconds to process 100,000 lines and put them into the database.


Time to review where the script was hanging up.   Look for the line that took the most time and we find:

     7516    6.378    0.001    6.378    0.001 {method 'commit' of 'sqlite3.Connection' objects}

In my script, I run a sql.commit() on every single INSERT into the database.   What if I only commit changes once I am done inserting?  This is a huge performance increase.   After this change, execution is almost cut in half:

3005019 function calls (3004892 primitive calls) in 10.767 seconds

There is a downside to this, if the script gets interrupted in the middle and there is no commit(), I will loose all data from that run.   In my case this is no big deal, I need performance over reliability, I can always rerun the import.

What is the next big thing?

    43541    7.247    0.000    7.247    0.000 {method 'execute' of 'sqlite3.Cursor' objects}
    36025    1.469    0.000    1.469    0.000 {method 'fetchone' of 'sqlite3.Cursor' objects}

My cursor and fetchone objects.   This is probably related to queries.   What can I do to queries to make them faster?  Let review what we are looking up.  The two major searches are:

    sqdb.execute('SELECT ip FROM leaseinfo WHERE date = ? AND ip = ?', t)
    sqdb.execute('SELECT mac FROM macinfo WHERE mac = ? AND wukey = ?', t)
 Lets index the data in the database to make these searches faster:

    sqdb.execute('''CREATE INDEX idx1 ON leaseinfo(date, ip)''')
    sqdb.execute('''CREATE INDEX idx2 ON macinfo(mac, wukey)''')

Another huge improvement.   Execution time was cut by another 79%:

         3005019 function calls (3004892 primitive calls) in 2.354 seconds

We started out with the script taking around 19.8 seconds.  By delaying commits to disk and indexing databases, execution time was dropped to 2.3 seconds.     This is an 89% improvement in execution time!


Now, the real file import that was taking >55 minutes.  After making the changes above:

time ./lease2sql.py -o -f ./infoblox.log
real 0m34.230s
user 0m33.944s
sys 0m0.237s

34 seconds to import.  Huge improvement.   

Friday, May 24, 2013

RedHat update problems because of bind conflict.

This is the second time all my RedHat servers stopped automatically updating.


The error when you run:

yum update


Error:


--> Processing Conflict: bind97-libs conflicts bind-libs
--> Processing Conflict: bind97-libs conflicts bind-libs
--> Processing Conflict: bind97-utils conflicts bind-utils
--> Finished Dependency Resolution
32:bind97-libs-9.7.0-17.P2.el5_9.1.x86_64 from rhel-x86_64-server-5 has depsolving problems
--> bind97-libs conflicts with bind-libs
32:bind97-utils-9.7.0-17.P2.el5_9.1.x86_64 from rhel-x86_64-server-5 has depsolving problems
--> bind97-utils conflicts with bind-utils
--> Running transaction check
---> Package kernel.x86_64 0:2.6.18-308.20.1.el5 set to be erased
--> Processing Conflict: bind97-libs conflicts bind-libs
--> Processing Conflict: bind97-libs conflicts bind-libs
--> Processing Conflict: bind97-utils conflicts bind-utils
--> Finished Dependency Resolution
32:bind97-libs-9.7.0-17.P2.el5_9.1.x86_64 from rhel-x86_64-server-5 has depsolving problems
--> bind97-libs conflicts with bind-libs
32:bind97-utils-9.7.0-17.P2.el5_9.1.x86_64 from rhel-x86_64-server-5 has depsolving problems
--> bind97-utils conflicts with bind-utils
Error: bind97-utils conflicts with bind-utils
Error: bind97-libs conflicts with bind-libs
You could try using --skip-broken to work around the problem
You could try running: package-cleanup --problems
package-cleanup --dupes
rpm -Va --nofiles --nodigest
 


We run automatic updates on many of our servers:

# automatically install updates
do_update = yes
# automatically download updates
do_download = yes
# automatically download deps of updates
do_download_deps = yes 



When errors like the ones above, automatic updates just break without notification.


To fix the above problem I checked which version of bind we had installed:

# rpm -qa | grep bind
bind-devel-9.3.6-20.P1.el5_8.5
bind-devel-9.3.6-20.P1.el5_8.5
bind-9.3.6-20.P1.el5_8.5
ypbind-1.19-12.el5_6.1
bind-libs-9.3.6-20.P1.el5_8.5
bind97-libs-9.7.0-10.P2.el5_8.4
bind97-utils-9.7.0-10.P2.el5_8.4
bind-libs-9.3.6-20.P1.el5_8.5
bind-utils-9.3.6-20.P1.el5_8.5


RedHat 5.x defaults version is 9.3.x.   There was no reason 9.7.x was installed, so we removed it:

yum remove bind97-libs 

Since the all the 9.7.x bind utils depend on this package, all 9.7.x packages will be removed.  

This fixed our problem.   Be sure to confirm you don't need the 9.7.x functionality before removing any packages! 



FYI: I never have problem with my Ubuntu servers.   This is the second time RedHat has failed me.



Monday, May 13, 2013

How to locate files in Ubuntu packages that have not been installed yet.

Many times I have a need to locate a package of a specific file that has not been installed yet.   One way to do this in Ubuntu is to install apt-file:
sudo apt-get install apt-file

Then update the search database by running:
sudo apt-file update


To locate a file run:
apt-file search DateTime.pm
  libatompub-perl: /usr/share/perl5/Atompub/DateTime.pm
  libchart-clicker-perl: /usr/share/perl5/Chart/Clicker/Axis/DateTime.pm
  libdata-faker-perl: /usr/share/perl5/Data/Faker/DateTime.pm
  libdatetime-perl: /usr/lib/perl5/DateTime.pm

While multiple packages contain DateTime.pm, it was fairly obvious which package I needed to install.

Wednesday, May 1, 2013

**ePN /usr/lib/nagios/plugins/check_hpasm: "Use of uninitialized value $_ in scalar chomp at (eval 21) line 340,".

We are running stock Nagios  on a Ubuntu 10.04 LTS server:


jemurray@nagios:/usr/lib/nagios/plugins$ /usr/sbin/nagios3 --version

Nagios Core 3.2.0
Copyright (c) 2009 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2009
License: GPL

We have been running the check_hpasm check script from http://labs.consol.de/nagios/check_hpasm/ for years with no problems.

After we installed and started running the F5 check script from: http://labs.consol.de/lang/en/nagios/check_nwc_health/


All our HP ASM checks were failing with the error:

**ePN /usr/lib/nagios/plugins/check_hpasm: "Use of uninitialized value $_ in scalar chomp at (eval 21) line 340,". 


After verifying we had all the latest versions of the check_* scripts, we found the built in Perl interpreter to be the problem.   To fix the problem, you have to force Nagios to use the system Perl interpreter instead of the Nagios built in one.   To do this locate the check_nwc_health and check_hpasm script and add the following line towards the top of the script:
# nagios: -epn

For example here is the first 10 lines of the script:


#! /usr/bin/perl -w
# nagios: -epn
package Nagios::MiniPlugin;
use strict;
use Getopt::Long qw(:config no_ignore_case bundling);
our @STATUS_CODES = qw(OK WARNING CRITICAL UNKNOWN DEPENDENT);


After this update, everything started working fine again.  

Monday, April 15, 2013

Are you ready for 802.11ac?

We are in a beta program testing out 802.11ac with one of our wireless vendors.   I fired up the spectrum analyzer during a few iperf sessions to see what a 802.11ac client looked like.   It is indeed 80Mhz wide.   

Time to start thinking about this...




Tuesday, March 26, 2013

Cpanel update failure during updatenow

We were getting the following error when the system runs the automatic update process (/usr/local/cpanel/scripts/upcp):

[20130325.005122] Testing if the newly downloaded RPMS can be installed without conflict
[20130325.005122] Testing RPM transaction
[20130325.005123] error: Failed dependencies:
[20130325.005123] MySQL conflicts with mysql-5.0.45-7.el5.i386
[20130325.005123] W Exit Code: 254
[20130325.005123] ***** FATAL: Test install failed: error: Failed dependencies:
[20130325.005123] MySQL conflicts with mysql-5.0.45-7.el5.i386
[20130325.005123] The Administrator will be notified to review this output when this script completes
[20130325.005123] ***** FATAL: Error testing if the RPMs will install: Died at /usr/local/cpanel/scripts/updatenow.static line 12520.
[20130325.005123] see http://go.cpanel.net/rpmcheckfailed for more information
[20130325.005123] The Administrator will be notified to review this output when this script completes
[20130325.005123] E Detected events which require user notification during updatenow. Will send iContact the log


Following the instructions listed here: http://go.cpanel.net/rpmcheckfailed

We find that exact error received.


 However, it says to remove the rogue RPM:

root@myserver [/etc/yum.repos.d]# rpm -qa|grep mysql-5
mysql-5.0.45-7.el5


My issue is this may remove files that are used in production right now.   The cPanel MySQL RPM's use the same file names, as the stock mysql, which should not have been installed in the first place.
root@myserver [/etc/yum.repos.d]# rpm -ql mysql-5.0.45-7.el5
/etc/ld.so.conf.d/mysql-i386.conf
/etc/my.cnf
/usr/bin/msql2mysql
/usr/bin/my_print_defaults
/usr/bin/mysql
/usr/bin/mysql_config
/usr/bin/mysql_find_rows
/usr/bin/mysql_tableinfo
/usr/bin/mysql_waitpid
/usr/bin/mysqlaccess
/usr/bin/mysqladmin
/usr/bin/mysqlbinlog
/usr/bin/mysqlcheck
/usr/bin/mysqldump
/usr/bin/mysqlimport
/usr/bin/mysqlshow
/usr/lib/mysql
/usr/lib/mysql/libmysqlclient.so.15
/usr/lib/mysql/libmysqlclient.so.15.0.0
/usr/lib/mysql/libmysqlclient_r.so.15
/usr/lib/mysql/libmysqlclient_r.so.15.0.0
.....extra lines cut....


In order to resolve this cPanel support suggested running:

rpm -e --nodeps --justdb mysql-5.0.45-7.el5.i386
If done correctly, it will just drop you back to the prompt. Then try your update again.  
What the above command does is not actually erase any RPM or dependencies from your server. It just hides the installed RPM from the RPM database so that the cPanel updater doesn't see it and won't have a conflict with it when it goes to install the proper version of MySQL that it wants to install.


Thanks cPanel support, as always you guys are great!

Saturday, March 16, 2013

OSX 10.8 gcc, compiling, xcode, etc...

After upgrading to OSX 10.8 (fresh install) gcc, clang, etc no longer existed

jemurray@dsg:~ $ gcc
-bash: gcc: command not found

In order to get all your open source compilers back you need to:


  1. Download the latest version of Xcode.   You can do that from the Apple App Store.
  2. Once Xcode is downloaded and installed, you need to install the Xcode Command Line Tools.
    1. Start Xcode
    2. Go to Xcode -> Preferences -> Downloads
    3. Click Install on Command Line Tools (see screen shot below)



That is it:

jemurray@dsg:~ $ which gcc
/usr/bin/gcc
jemurray@dsg:~ $ which clang
/usr/bin/clang
jemurray@dsg:~ $ ls -ald /usr/include
drwxr-xr-x  281 root  wheel  9554 Mar 15 13:53 /usr/include


Followers

Blog Archive

Contributors