Ubuntu – Automating virtual machine installation using network preseeds

Virtual machines are very useful for testing. I often use them to verify changes to software, without messing up the local environment. Due to laziness I use VirtualBox and install Ubuntu official ISOs on them, rather than something more elegant/complicated such as kvm, lxc containers or chroots. This replicates an actual desktop environment pretty closely so is ideal for reporting bugs and validating that fixes to software work as expected.

Taking a virtual machine to a point where it’s mostly usable is a bit involved. I launched the desktop ISO, did the manual install procedure, rebooted, installed the VirtualBox extensions so I could mount the host’s drives, did some group changes, rebooted again… this is getting a bit tiring!

I had a quick look at Vagrant to see if it could somehow ease the task. It’s very interesting but didn’t really work in this case, as the virtual machine still has to be set up the way I describe before being able to package and then use it. What I’m after, really, is a way to set up a VM from scratch, just by doing the installation and adding a few extra packages.

This is what preseeding does, but up until now I had only played with local preseeds, baked into the ISO image. I imagined being able to load a preseed from the network would be difficult to set up quickly, and on a personal workstation, which is what would best fit my use case.

Turns out that virtualbox and a simple python module make this very easy. With the default configuration (NAT networking), a virtualbox VM will get an IP address through DHCP, and it will be able to reach the host’s public IP address. So as long as we configure the Ubuntu installer  correctly and have something serving that file, things are very easy. One of the parts I like about this is that experimenting with this is as easy as changing the local preseed and rebooting the VM. About the only cumbersome part is typing the kernel parameters every time, but since there’s only three of them to type/change, this is not as bad as it sounds.

  1. Put your preseed files in a directory (called, for instance, preseed.cfg).
  2. Change to this directory and run python -m SimpleHTTPServer. This starts a miniature HTTP Server on port 8000.
  3. If you like, verify that the preseed is served properly: wget http://:8000/preseed.cfg
  4. Set up the virtual machine, point it to the Ubuntu installation CD, start it.
  5. When you get the keyboard and human icon, press any key.
  6. Move to “install Ubuntu” but don’t press Enter.
  7. Press F6 to access the “advanced mode”. At this point we’re modifying the kernel command line.
  8. Go to the beginning, delete the “file=” portion.
  9. Add “auto url=http://:8000/preseed.cfg”.
  10. Replace “only-ubiquity” with “automatic-ubiquity”.
  11. Press Enter
  12. Sit back and relax while the virtual machine gets installed.

This fits the bill perfectly for me, it removes the manual steps in setting up a testing VM (which I don’t need to keep afterwards, so I can just delete it and recreate with the same procedure), allows for easy experimentation and customization, and doesn’t use a lot of strange technologies or components.

Here’s a link to a sample, basic preseed file. You can customize mainly the late_command (rather, the success_command for ubiquity) and anything else you like. The installation-guide-amd64 package has more details and sample preseed files.

Note that for server installations the kernel command line will be a bit different:

  • No need to add automatic-ubiquity.
  • You DO need to add the “auto url=blahblah” part.
  • For it to be 100% automated, you need to specify a few parameters that in debian-installer are requested *before* the preseed is loaded. Add these: debconf/priority=critical locale=en_US console-setup/ask_detect=false console-setup/layoutcode=us netcfg/choose_interface=auto
  • Note that for debian-installer, the late_command is used as opposed to the ubiquity/success_command.

Reference

Debian preseeding guide

Ubiquity automation

 

Video conversion for iPhone with avconv

avconv replaces the venerable ffmpeg. It can be used to convert videos for the iPhone quite easily.

then run this script:

Another example. This uses time to calculate elapsed time, also nice and ionice to try to reduce impact on system resources. It forces downsampling to two audio channels (-ac 2), useful if the source audio stream is in e.g. 5.1 format.

 

A final example which forces a specific aspect ratio. The source video had the correct pixel dimensions but a bad aspect ratio was encoded in the original file (and was carried over to the recoded one), making it look squished.

Vim and the X clipboard

Usually when I needed to paste stuff from a text file into a GUI program (most commonly, the browser), I resorted to opening the text file in gedit and copying/pasting from there. Using the X clipboard by selecting text with the mouse kinda worked, but it’s subject to Vim’s visual representation of the text, which may include unwanted display-related breaks. So using gedit was the easiest, but also awfully kludgy solution.

I did some research and learned that vim does have direct access to the X clipboard. I tried the commands they mention (basically “+y to yank selected text, then I tried to paste in a GUI application; or “+p to paste from the current X clipboard). They didn’t work. My installed version of Vim in Ubuntu lacked the xterm_clipboard setting. I was in despair!

Then I came across this bug report in Launchpad. Upon reading it I realized that it was as simple as installing vim-gtk. I had never considered this, as it includes a graphical Vim version which I have absolutely no use for. However the bug report mentions that it also includes a text version of vim compiled with X clipboard support. So I installed, fired up Vim, and the feature works well!

I can now have a buffer with long lines, with :set wrap and :set linebreak, which would be afwul if I cut/pasted it with the mouse. I can select text using vim commands and just yank it into the + register, and it’s instantly available in the X clipboard. Bliss!

 

Tirando el dinero a la basura

A la hora de votar acuérdense de cómo en los dos últimos sexenios (PAN) simplemente el dinero se tira a la basura, con resultados enteramente dudosos (Estela de Luz, sede del Senado) o de plano sin absolutamente nada qué mostrar (RENAUT, simplemente se va a “destruir la base de datos”, es decir que el gasto se hizo y no se tiene ningún producto como reusltado).

Recuerden cómo los panistas dijeron que la manera de eficientar al gobierno era manejarlo como una “empresa” y enfocarse en costo-beneficio, eficiencia y otras “medidas”. Ahora piensen lo que les pasaría a ustedes si toman el dinero de su empresa y lo QUEMAN simplemente. Ninguna empresa puede aguantar así, y ningún ejecutivo que hiciera eso conservaría su trabajo.

El gabinete de los viejitos

Cuando sea hora de votar, acuérdense de esto:

http://www.jornada.unam.mx/2012/02/28/politica/014n1pol?partner=rss

Josefina Vázquez Mota, afirmó que cuando ve a otro aspirante que nombra a su gabinete, pienso que suman como mil 500 años de edad, y que al observar cómo en otro partido se integra a su consejo político a ex gobernadores, algunos con historias terribles, pienso que suman como mil 500 años de prisión.

Acuérdense de estos comentarios a la hora de ir a votar, acuérdense de cómo desprecia a la gente simplemente por su edad, cómo para ella la experiencia no vale nada, y piensen si así descalifica y menosprecia a sus colegas políticos, ¿qué hará con los planes de apoyo a la tercera edad?.

Elecciones – Desconocidos vs. Superestrellas

¿Por quién prefieres votar? ¿Por Vázquez Mota y su equipo de “desconocidos” que van a seguir hundiendo al país tratándolo como si fuera una empresa?

¿O por este equipo de superestrellas de la ciencia, el arte, la política y la economía?

Y la neta, si no los conoces, ve investigando quienes son. La decisión es tuya.

http://www.jornada.unam.mx/2012/02/17/economia/008o1eco

Ubuntu and Juju with local providers

Want to play with Ubuntu’s awesome Juju but don’t want to get into the hassle of getting EC2 configured?

https://juju.ubuntu.com/

It’s actually pretty easy to set up a local provider to experiment with this.

You need to be running Ubuntu 12.04 (yes, it’s not released yet but you can use the beta version or daily  images). Oh, and you can install this on a virtual machine if you really don’t want Juju to mess with your actual system.

Make sure to have a valid SSH key, if you don’t have one, create it with

Then

Once the packages are installed, run

this adds you to the libvirtd group at runtime.

Then run juju bootstrap. Juju will complain about a config file or something. Ignore it! and then edit .juju/environments.yaml and replace everything on that file with this:

The admin-secret is a MD5 random key, you should probably generate your own with something like this:

Then finally it’s time to bootstrap things:

This will exit pretty quickly, but things are not ready yet. Note that it will take a few minutes to get packages and actually prepare the nodes.

Once your juju is bootstrapped you can follow the rest of the steps here:

https://juju.ubuntu.com/docs/user-tutorial.html#bootstrapping

Find and -exec and why doesn’t basename work in there?

Ever wonder why this doesn’t work to rename all your isos to .nrg?

The thing is, the backticks are a bash construct, and whatever you specify to -exec does *not* get shell-expanded. So it’s being passed verbatim, with the only substitution being the {} for the found filenames.

But this works! (though quoting may pose some challenges). This works because it *is* being processed through bash, so all the normal expansion and shell tricks will work.

Notifications – during and after launching

When I launch a long-running process I like to forget about it, but how do I know when it’s finished?

You can of course have it send an email after it finishes:

For this to work, it’s very useful to have ssmtp configured, so you have a sane, working local SMTP agent.

You can also send the notification only if the command succeeds:

OK, so you forgot to add the notification to your initial command line. You can use a loop to monitor a particular process and notify you when it’s done.

In this case I’ll be monitoring an instance of netcat. Determining the process name is up to you 🙂 The delimiters $ and ^ look for the executable names only.

The while loop will run while the process exists; once the process disappears the loop continues with the next instruction in the line, which is popping up an alert on the desktop and then sending an email. So if I’m not glued to the desktop, I’ll still get an email when this is done.

while pgrep $nc^; do sleep 5; done; alert; (echo “finished” |mail -s “finished” you @somewhere.com)

find’s printf action

If you use find, it outputs full paths, which may not always be desirable. Turns out find has a -printf action with which you can do  niceties such as outputting plain filenames (as if you’d used basename on them, but this means one less command on your pipeline):

The -printf command has a lot of formatting variables and possibilites! give it a try, look at the man page for more information.

ASCII video rendering

If you’re a CLI jockey you may enjoy looking at a nice ASCII rendering of your face via your webcam:

mplayer -vo caca tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0

Or to watch your favorite video in ASCII rendering:

vlc –vout caca some-file.avi

 

 

Random picking without repetition

So the problem was to draw people at random from a list. The list is contained in a leads.txt text file, one per line.

This nifty one-liner will output a randomly-picked person from that file every time it’s invoked. it’ll then remove the name from the file so it doesn’t get repeated again.

It can be shortened by changing shuf |head 1 to shuf -h 1.

If you’d rather avoid deleting already-chosen entries from the file, this version just comments the names it picks:

Building Debian/Ubuntu packages with sbuild

Many of the on-line instructions and tutorials are quite complicated. Why? It was easy for me:

To build a virtual machine:

this will create a schroot in /var/lib/schroots/precise-i386. Note how it appends the architecture to the schroot name. Also note that the first time you run mk-sbuild, it’ll show you a configuration file and configure your environment. I didn’t change anything in the config file, I used it “as it was”. When it prompts you to log out, do it, otherwise things won’t work.

OK now you want to build a package using your chroot with sbuild:

This will build the package on precise for ALL available architectures. Note that -d is just “precise”; the -A flag will tell sbuild to build architecture: any packages for all available architectures (so if you have amd64 and i386 chroots, it’ll do the right thing and build two packages).

If you want to build arch-specific packages:

This will magically build for the given architecture (i386). Note that arch: any packages will also be built.

You can also specify the arch as a parameter (but then you have to leave it out of the -d name):

This will not work: