Akom's Tech Ruminations

Various tech outbursts - code and solutions to practical problems

Linux Jenkins Swarm Slaves on Windows using Puppet

Posted by Admin • Monday, October 12. 2015 • Category: DevOps, Linux
Jenkins Swarm plugin is great, and instrumentation for Linux is fairly well-known, but what about Windows? Here is one approach for setting it up as a windows sevice (what we want Puppet to do):
  1. Download Swarm jar
  2. Download winsw from Kohsuke
  3. Rename winsw to jenkins_swarm.exe
  4. Create jenkins_swarm.xml
  5. Run jenkins_swarm.exe install
  6. Start service
Now for the details

Continue reading "Jenkins Swarm Slaves on Windows using Puppet"

Linux Jenkins Windows Slaves cannot install JDK

Posted by Admin • Monday, October 12. 2015 • Category: DevOps, Linux
Despite making the jenkins slave user a local administrator. The slave runs tools\hudson.model.JDK\JDK_1.7\jdk.exe (or 1.6), and fails (see the extended post body for the log).

If you're seeing Error 1722.There is a problem with this Windows Installer package, then most likely you have another (newer?) version of this JDK installed system-wide. Uninstall from Control Panel, try again.

Continue reading "Jenkins Windows Slaves cannot install JDK"

Linux Webex in Ubuntu 15.04

Posted by Admin • Monday, September 28. 2015 • Category: Linux
The following was required to get shared screens to display:
  1. Using openjdk 7 (although I have 7 and 8 installed, 8 is default, and I removed oracle java - but the browser plugin still runs 7)
  2. Follow Option 1 from http://askubuntu.com/a/623397/238077
  3. If you are missing libxmu6, install the 32 bit version: sudo apt-get install libxmu6:i386 (this is actually mentioned in the answer, but I missed it)
  4. Firefox should now support webex

Linux Getting VMware vSphere 5.5 Server Console to work on Ubuntu Linux 15.04

Posted by Admin • Wednesday, September 16. 2015 • Category: DevOps, Linux
As everyone who uses this product eventually finds out, VMware vSphere (5.5) browser-based web client will generally work in Linux (at least in chrome), but opening a server console is nearly impossible. The web client requires flash 11.5, and of course there is no linux flash above 11.2...

After a lot of dancing I managed to get it working in Firefox 40.0.3 on Ubuntu 15.04. Here are are the steps in a nutshell:
  1. Clear out your ~/.mozilla ... True, you may not need to, but I did. Start firefox once, then exit.
  2. Install pipelight
  3. Enable flash in pipelight (I wound up doing this globally and for current user, not sure which one did it
  4. Install the VMware client integration plugin (if you did before, uninstall and reinstall). That's the download you get on the bottom of the vSphere login page
  5. Start firefox and see if it works. If not, fiddle with the plugins (under extensions). You may see two versions of flash. I was not able to disable 11.2 from the UI without also disabling version 18 from pipelight - so I wound up deleting /usr/lib/mozilla/plugins/flashplugin-alternative.so (but you can also apt-get remove flashplugin-installer)
  6. Explicitly go to extensions again and switch all VMware items to "Always Activate"
  7. Restart firefox over and over throughout the process :-)


Good luck



Update: After some system updates, the pipelight plugins stopped appearing in Firefox, and needless to say nothing worked. Deleting ~/.mozilla again seemed to resolve it (You then have to switch all the plugins to "Always Activate" again).

Linux Maven toolchains and puppet

Posted by Admin • Monday, August 31. 2015 • Category: DevOps, Linux
For today's trick we'll be setting up maven toolchains.xml files on Jenkins slaves that are managed by puppet. One option for doing this is to use the Jenkins Config File Provider plugin - but all that really does is push a predetermined file to the slaves before running the job (now if it only worked together with the Jenkins tool provider data...) In our case, we have Jenkins slaves with different OS versions and thus different toolchain locations - for example, minor JDK releases differ across OS's and thus the paths are not the same.

Instead, the plan is to do two things:
  1. Create a puppet custom fact with installed tool info
  2. Use it from a puppet template
Here is how

Continue reading "Maven toolchains and puppet"

Linux Getting ETVnet.com to play video on Ubuntu

Posted by Admin • Friday, June 12. 2015 • Category: Linux
In Ubuntu 15.04 it already works in Firefox as long as you have vlc installed, and you select /usr/bin/vlc the first time Firefox asks you what to do. Not so much in Chrome... Chrome uses xdg-open to determine what to launch. By default, xdg-open has no idea what to do with a mms://1.2.3.4/something.wmv style URL, so we need to set up a protocol action in xdg, as per http://askubuntu.com/questions/190895/how-to-change-what-xdg-open-does-with-ssh-userip-liniks :
xdg-mime default vlc.desktop x-scheme-handler/mms


UPDATE: I find that vlc does not gracefully recover from eTVnet's "glitches" - sound stops playing but video continues. Surprisingly, Totem does not have this issue, so I swtiched:
xdg-mime default totem.desktop x-scheme-handler/mms

Linux Simple puppet update-alternatives

Posted by Admin • Thursday, June 11. 2015 • Category: DevOps, Linux
This is a quick and dirty interface to update-alternatives on Centos/Redhat/Ubuntu for puppet. Seems to work well and doesn't require any modules.
Usage example: alternatives_update { 'java': versiongrep => '1.8' }

class my_alternatives {


  # Manipulates alternatives using update-alternatives.
  # Supports RHEL, Centos and Suse.
  # Ubuntu not tested (yet).
  #
  # If multiple matches are available, picks the first one.
  #
  # There is rudimentary alternatives support in the java class,
  # but it's rather limited and doesn't support most platforms and java versions.
  define update (
    $item = $title,   # the item to manage, ie "java"
    $versiongrep,     # string to pass to grep to select an alternative, ie '1.8' (1.8.*openjdk would also work)
    $optional = true,  # if false, execution will fail if the version is not found
    $altcmd   = 'update-alternatives' # command to use
  ) {

    case $::osfamily {
      'RedHat','SuSE': {


        if ! $optional {
          # verify that we have exactly 1 matching alternatives, unless it's optional
          exec { "check alternatives for ${item}":
            path    => ['/sbin','/bin','/usr/bin','/usr/sbin'],
            command => "echo Alternative for ${item} version containing ${versiongrep} was not found, or multiple found ; false",
            unless  => "test $(${altcmd} --display ${item} | grep '^/' | grep -- '$versiongrep' | wc -l) -eq 1",
            before  => Exec["update alternatives for ${item} to ${versiongrep}"],
          }
        }

        # Runs the update alternatives command
        #  - unless it reports that it's already set to that version
        #  - unless that version is not found via grep
        exec { "update alternatives for ${item} to ${versiongrep}":
          path    => ['/sbin','/bin','/usr/bin','/usr/sbin'],
          command => "${altcmd} --set ${item} $( ${altcmd} --display ${item} | grep '^/' | grep -- '$versiongrep' | head -n 1 | sed 's/ .*$//' ) ",
          unless  => "test -x \"$(${altcmd} --display ${item} | grep 'currently points' | grep -- '$versiongrep' | awk '{print \$NF}')\"",
          onlyif  => "${altcmd} --display ${item} | grep '^/' | grep -- '$versiongrep' ", # check that there is one (if optional and not found, this won't run)
        }

      }

      # Leave Ubuntu alone, this probably won't work there anyway


    }
  }

}
 

Linux Getting a systemd unit to read your .bashrc file for its environment

Posted by Admin • Thursday, June 11. 2015 • Category: Linux
Although it'd be nice to have all of your services not rely on their shell environment, sometimes it is unavoidable. In my case, some of the systems are still on init.d (redhat <=65, centos <=65, etc), but some are on systemd (7.0's, suse 12, etc). The old init scripts rely on the .bashrc file. In order to make the service run consistently on both flavors, I had to teach systemd to read the user's bashrc file. This is what it looks like.


[Service]
Type=simple
User=some_user
# Process the normal environment files for this user by starting a login shell
# and outputting it all in a temp file.  This makes it compatible
# with the non-systemd init scripts that still rely on .bashrc
ExecStartPre=/bin/bash --login -c 'env > /tmp/.magic-environment-file'
ExecStart=SERVICE_COMMAND
EnvironmentFile=-/tmp/.magic-environment-file
 
The "-" before the filename seems to make systemd ignore the error if the file is not there (for example, on the first run of the pre command). Now, obviously, you could just set your ~some-user/.bashrc as the EnvironmentFile - but your mileage will vary. systemd will not interpolate variables and it will ignore lines starting with "export"...

Continue reading "Getting a systemd unit to read your .bashrc file for its environment"

Linux Wifi doesn't work after resume from suspend in Ubuntu 15.04 (Dell Latitude E4310)

Posted by Admin • Sunday, May 17. 2015 • Category: Linux
Although toggling the hardware Wi-Fi switch usually fixed this issue, it was decidedly annoying for the non-technical user of this laptop. Googling for a well-known solution turned up several, but none of these worked. I found that either running
nmcli r wifi off
nmcli r wifi on
usually worked, so I tried to stick that into /etc/pm/sleep.d - but that script was never used. I then discovered that in 15.04 Ubuntu switched to systemd, which requires a service file in /etc/systemd/system/ In the end, it turns out that the problem is intermittent, therefore it is not possible to simply toggle networking blindly and hope for the best. Half the time it will not work. The ultimate solution turned out to be a combination of a toggle and a check, and both could be done via network manager, as follows:

Continue reading "Wifi doesn't work after resume from suspend in Ubuntu 15.04 (Dell Latitude E4310)"

Linux Building RPMs for an older version of CentOS

Posted by Admin • Thursday, November 20. 2014 • Category: Linux
If you build RPMs on Centos 6.X (6.5 in my case) and then try to install them on Centos 5.X (5.10 in my case), bad things happen. Ironically, bad things happen even though my RPM contains a single jar file, and is thus entirely platform independent. Here is what I see:
Running rpm_check_debug
ERROR with rpm_check_debug vs depsolve:
rpmlib(FileDigests) is needed by my-rpm-1.0.x86_64
rpmlib(PayloadIsXz) is needed by my-rpm-1.0.x86_64
If you google for a solution, most people suggest running a virtual with Centos 5 just so you can build the RPMs, but this is apparently not necessary in this simple case, as you can simply specify a few flags. Basically, Centos 5 can't decompress the default archive format, and doesn't support the new digest algorithm.

Since I am using maven-rpm-plugin to build RPMs, my modifications look as follows:

                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>rpm-maven-plugin</artifactId>
                    <version>2.1-alpha-3</version>
                    <configuration>

                      <!-- skipping irrelevant items -->

                        <defineStatements>
                            <!-- don't strip jar files, it takes forever and is useless -->
                            <defineStatement>__os_install_post %{nil}</defineStatement>
                            <!-- for Centos6 -> Centos5 forwards compatibility -->
                            <defineStatement>_source_filedigest_algorithm md5</defineStatement>
                            <defineStatement>_binary_filedigest_algorithm md5</defineStatement>
                            <defineStatement>_source_payload w9.bzdio</defineStatement>
                            <defineStatement>_binary_payload w9.bzdio</defineStatement>
                        </defineStatements>

                   </configuration>
                </plugin>
 
If you're using fpm or rpmbuild, you can just take these lines and make them %define's in your spec.

Linux Backing up cPanel 11 hosted account with wget and dav/rsync

Posted by Admin • Thursday, August 21. 2014 • Category: Linux
I want to back up my hosting account regularly by retrieving everything onto my box somewhere else (my home server). By regularly, I mean every day. I want this to happen automatically. cPanel makes that hard to do, but there is always a way to script things.

First, let's break down what we want to back up:
  1. Files
  2. Databases and email forwarders
Files are fairly easy. Obviously we don't want to use the full backup functionality of cPanel because we'd be transferring your entire storage space each and every time, even if nothing has changed. In order to do it efficiently, you have a few options:
  1. FTP account using recursive wget (create an FTP account with required access and teach an ftp client of your choice to recursively transfer everything. Hopefully this client skips unchanged files)
  2. WebDAV using rsync (this is what I'm using). Just mount, back up, unmount
Databases and Email stuff is not as easy, as we do have to log into cPanel. The trick to logging in to cPanel is:
  1. Submit your login to the log-in page and save cookies
  2. Parse the resulting file, find the backup link which includes your session name in the URL and hit that
  3. Only accept .gz files (DB backups and email stuff), but avoid hitting /logout, and don't start spidering the entire website
So, how do we do this? Here is the plan

Continue reading "Backing up cPanel 11 hosted account with wget and dav/rsync"

Linux mssh Bash completion

Posted by Admin • Friday, February 28. 2014 • Category: Linux
Clusterssh (cssh) is great, but I was getting a little fed up with unmanageable terminal windows. They either go all over my monitors or get lost, and they are hard to move and resize.

So I switched to mssh, which solves all that because all the terminals are in one window.... but, it doesn't read my /etc/clusters file! In fact, nobody seemed to even know what file it does read.

What to do? First of all, for the record, it reads its aliases (-a) from ~/.mssh_clusters This file is exactly like /etc/clusters, except for a colon, like so:
alias1: host host host
alias2: host host host

Now that we got that straight, let's make a bash completion file

Continue reading "mssh Bash completion "

Linux Upgrading pfSense fails due to low disk space

Posted by Admin • Tuesday, October 22. 2013 • Category: Linux
I have an embedded installation of pfSense 2.0.2-RELEASE running from a 512MB compact flash card via an IDE adapter. At the time this was what I had on hand, and it seemed like plenty of space - but now that I tried to upgrade to 2.1, the process failed because there wasn't enough room to download the new image...

I tried all sorts of ways including both webconfigurator upgrade and trying to scp the image to the box, but either way there simply wasn't enough space. "OK" I said, "let me install 2.1 on a new (larger) card and restore the config.xml" .... bad idea. The firewall becomes unbootable when you do this - I think that no upgrade processing is performed on the configuration, it is assumed to be a match for your software version. Not to mention that I tried a 4GB card, but the old Pentium III running the firewall can't handle anything over 2GB!

Ultimately, this is the process:
  1. Download and install the same version as what you're running on the larger card
  2. Restore your config.xml backup onto the new instance of pfSense. You can do this through webconfigurator, or if you are (like me) running on a temporary rig without multiple NIC's, you can scp it over manually
  3. Confirm that the system boots, but don't make any changes (to the interfaces, for instance). Put the card into your real system
  4. Now you can use the auto-updater as normal, and your old config.xml will be upconverted

Linux amixer toggle mutes but does not unmute

Posted by Admin • Monday, July 22. 2013 • Category: Linux
It seems that the common approach to mute the audio in Ubuntu 13.04 is amixer -q sset Master toggle ... which works great when muting, but fails to unmute. This command is, for example, the default for openbox's handling of XF86AudioPlay shortcut.

The problem, it would appear, is that muting the Master channel causes other channels (eg Headphone and Front on my machine) to be muted as well. I have no idea why this works this way and why it doesn't undo what it did... and, frankly, I don't really care that much. It's much easier to code around this, see the following workaround script:

Continue reading "amixer toggle mutes but does not unmute"

Linux motion drops MySQL support, but you can get around it with a shell script

Posted by Admin • Monday, September 24. 2012 • Category: Linux
I use motion as a surveillance system for a sizable office (9 cameras). Motion is good. But the current package of motion in Ubuntu 12.04 is no longer compiled with MySQL support (used to have it in 10.04). Compiling motion from source really didn't sound appealing... but I do want to add events to a database for my own in-house motion history browser. What to do?

Motion supports command execution on events, such as on_movie_end and on_picture_save... which makes it really easy to do a one-liner like this:
on_movie_end bash -c "echo insert into security(camera, filename, frame, file_type, time_stamp, event_time_stamp, event_id) values('%t', '%f', '%q', '%n', '%Y-%m-%d %T', '%C', '%v') | mysql -uUSER -pPASS"
if you'd like to do something fancier, you can break it out into a shell script:

#!/bin/bash

logger -t test "$0 executing with $*"
sql="insert into security(camera, filename, frame, file_type, time_stamp, event_time_stamp, event_id) values('$1', '$2', '$3', '$4', '$5', '$6', '$7')"
echo $sql | mysql -u$U -p$P motion
 
and invoke like this:
on_movie_end U=USER P=PASS bash /usr/local/sbin/motion-insert-event.sh '%t' '%f' '%q' '%n' '%Y-%m-%d %T' '%C' '%v


(same for on_picture_save, if needed). You can also daisy-chain commands with semicolon if you have multiple. Also, note that the query above is for my table, yours may have different columns. The rest are my specific examples of usage:

Continue reading "motion drops MySQL support, but you can get around it with a shell script"