Markdown

Categories: Uncategorized

Projects on github will show a README or README.md file directly on the project page. This is a good place to give some introduction or quick instructions for your project. This supports Markdown, which allows you to craft a README that will both be readable when seen in plain text, and will render nicely when seen directly in github.

Here’s a handy Markdown syntax reference and tutorial. Also, at some point I needed clarification on how to make nested lists, which I found in StackOverflow. There’s a wealth of Markdown-related information on the web!

Two useful tidbits. To render a markdown document to HTML, for previewing so you don’t have to upload stuff to github just to see what your README will look like,

apt-get install  python-markdown

and then run

markdown_py

on your README.md.

Also, vim supports markdown and will do its best to help you, but one unhelpful thing is its insistence to render underscores (_) in inverted text (as it assumes it’s the beginning of an underlined section). Just a warning 🙂

ifconfig.me – Programmatically-usable external IP finder

Categories: English Geeky

You’re probably familiar with whatismyip services. One problem with these is that they wrap the IP data in a visually-pleasing but impossible to parse smorgasboard of HTML.

I needed to determine external IP for a text-only host with somewhat limited tools, and I came across the wonderful ifconfig.me page. This basically does the same as whatismyip but with lightweight and easily parsable output. This, for instance, returns *only* the system’s external or public IP address:

curl http://ifconfig.me/ip

Several endpoints are provided for you to poke at your connection’s externally-visible information. For the ultimate in parsability and machine-readability, a complete json dump is provided here: http://ifconfig.me/all.json.

Recently I’ve had better success with http://ifconfig.co which works mostly similarly to ifconfig.me but is a bit faster.

Recording terminal sessions – script and friends

Categories: Uncategorized

Most of us have wanted, at some time or another, to share an entire “session” on the terminal. I mean showing a log of the commands we typed, their output, and so on, to illustrate a procedure or clarify a concept.

The “script” command has long been a tool of choice for this. To use it, just type:

script

this will seem to start a new shell process and start recording everything you type and everything that other commands output in response. To stop recording, just type “exit”. A file called typescript will be created with your session. If you want, you can call script with a parameter (script your-file.log) to log to that file instead.

A newer implementation of the concept is ttyrec. It behaves mostly the same as script, but instead of producing openable text files, it logs keystrokes and timing information. You need to play these back using ttyplay, but the result is pretty amazing because you can see the keystrokes being played back as they were typed. You can also instruct ttyplay to just dump the session to a plain text file, which mimics script’s behavior.

Finally, to take this concept one step further, have a look at showterm, which does essentially the same as ttyrec but automatically saves and uploads the session to a website for you to share with people. Think of it as an interactive pastebin.

Ubuntu – Automating virtual machine installation using network preseeds

Categories: Uncategorized

Virtual machines are very useful for testing. I often use them to verify changes to software, without messing up the local environment. Due to laziness I use VirtualBox and install Ubuntu official ISOs on them, rather than something more elegant/complicated such as kvm, lxc containers or chroots. This replicates an actual desktop environment pretty closely so is ideal for reporting bugs and validating that fixes to software work as expected.

Taking a virtual machine to a point where it’s mostly usable is a bit involved. I launched the desktop ISO, did the manual install procedure, rebooted, installed the VirtualBox extensions so I could mount the host’s drives, did some group changes, rebooted again… this is getting a bit tiring!

I had a quick look at Vagrant to see if it could somehow ease the task. It’s very interesting but didn’t really work in this case, as the virtual machine still has to be set up the way I describe before being able to package and then use it. What I’m after, really, is a way to set up a VM from scratch, just by doing the installation and adding a few extra packages.

This is what preseeding does, but up until now I had only played with local preseeds, baked into the ISO image. I imagined being able to load a preseed from the network would be difficult to set up quickly, and on a personal workstation, which is what would best fit my use case.

Turns out that virtualbox and a simple python module make this very easy. With the default configuration (NAT networking), a virtualbox VM will get an IP address through DHCP, and it will be able to reach the host’s public IP address. So as long as we configure the Ubuntu installer  correctly and have something serving that file, things are very easy. One of the parts I like about this is that experimenting with this is as easy as changing the local preseed and rebooting the VM. About the only cumbersome part is typing the kernel parameters every time, but since there’s only three of them to type/change, this is not as bad as it sounds.

  1. Put your preseed files in a directory (called, for instance, preseed.cfg).
  2. Change to this directory and run python -m SimpleHTTPServer. This starts a miniature HTTP Server on port 8000.
  3. If you like, verify that the preseed is served properly: wget http://:8000/preseed.cfg
  4. Set up the virtual machine, point it to the Ubuntu installation CD, start it.
  5. When you get the keyboard and human icon, press any key.
  6. Move to “install Ubuntu” but don’t press Enter.
  7. Press F6 to access the “advanced mode”. At this point we’re modifying the kernel command line.
  8. Go to the beginning, delete the “file=” portion.
  9. Add “auto url=http://:8000/preseed.cfg”.
  10. Replace “only-ubiquity” with “automatic-ubiquity”.
  11. Press Enter
  12. Sit back and relax while the virtual machine gets installed.

This fits the bill perfectly for me, it removes the manual steps in setting up a testing VM (which I don’t need to keep afterwards, so I can just delete it and recreate with the same procedure), allows for easy experimentation and customization, and doesn’t use a lot of strange technologies or components.

Here’s a link to a sample, basic preseed file. You can customize mainly the late_command (rather, the success_command for ubiquity) and anything else you like. The installation-guide-amd64 package has more details and sample preseed files.

Note that for server installations the kernel command line will be a bit different:

  • No need to add automatic-ubiquity.
  • You DO need to add the “auto url=blahblah” part.
  • For it to be 100% automated, you need to specify a few parameters that in debian-installer are requested *before* the preseed is loaded. Add these: debconf/priority=critical locale=en_US console-setup/ask_detect=false console-setup/layoutcode=us netcfg/choose_interface=auto
  • Note that for debian-installer, the late_command is used as opposed to the ubiquity/success_command.

Reference

Debian preseeding guide

Ubiquity automation

Video conversion for iPhone with avconv

Categories: Uncategorized

avconv replaces the venerable ffmpeg. It can be used to convert videos for the iPhone quite easily.

apt-get install avconv libavcodec-extra-53 libx264-123 x264

then run this script:

avconv -i input-file.mp4 \
    -vcodec libx264 -vprofile high \
    -t 30 \
    -preset slow \
    -b:v 1000k -maxrate 1000k \
    -bufsize 2000k -vf scale=1136:474 \
    -acodec libvo_aacenc -b:a 192k output_file.mp4

Another example. This uses time to calculate elapsed time, also nice and ionice to try to reduce impact on system resources. It forces downsampling to two audio channels (-ac 2), useful if the source audio stream is in e.g. 5.1 format.

time ionice -c 3 nice -20  avconv  -i whatever.avi  \
-vcodec libx264 -vprofile high -preset slow -b:v 1000k\
 -maxrate 1000k -bufsize 2000k  -acodec libvo_aacenc \
-ac 2 -ab 112k output.mp4

A final example which forces a specific aspect ratio. The source video had the correct pixel dimensions but a bad aspect ratio was encoded in the original file (and was carried over to the recoded one), making it look squished.

avconv  -i input.avi  -vcodec libx264 -aspect 16:9 \
-vprofile high -preset slow -b:v 1900k -maxrate 1900k \
-bufsize 3800k  -acodec libvo_aacenc -ac 2 -ab 112k output-iphone.mp4

Vim and the X clipboard

Categories: Uncategorized

Usually when I needed to paste stuff from a text file into a GUI program (most commonly, the browser), I resorted to opening the text file in gedit and copying/pasting from there. Using the X clipboard by selecting text with the mouse kinda worked, but it’s subject to Vim’s visual representation of the text, which may include unwanted display-related breaks. So using gedit was the easiest, but also awfully kludgy solution.

I did some research and learned that vim does have direct access to the X clipboard. I tried the commands they mention (basically "+y to yank selected text, then I tried to paste in a GUI application; or "+p to paste from the current X clipboard). They didn’t work. My installed version of Vim in Ubuntu lacked the xterm_clipboard setting. I was in despair!

Then I came across this bug report in Launchpad. Upon reading it I realized that it was as simple as installing vim-gtk. I had never considered this, as it includes a graphical Vim version which I have absolutely no use for. However the bug report mentions that it also includes a text version of vim compiled with X clipboard support. So I installed, fired up Vim, and the feature works well!

I can now have a buffer with long lines, with :set wrap and :set linebreak, which would be afwul if I cut/pasted it with the mouse. I can select text using vim commands and just yank it into the + register, and it’s instantly available in the X clipboard. Bliss!

Building Debian/Ubuntu packages with sbuild

Categories: Uncategorized

Many of the on-line instructions and tutorials are quite complicated. Why? It was easy for me:

sudo apt-get install sbuild

To build a virtual machine:

mk-sbuild --distro=ubuntu --arch=i386 precise

this will create a schroot in /var/lib/schroots/precise-i386. Note how it appends the architecture to the schroot name. Also note that the first time you run mk-sbuild, it’ll show you a configuration file and configure your environment. I didn’t change anything in the config file, I used it “as it was”. When it prompts you to log out, do it, otherwise things won’t work.

OK now you want to build a package using your chroot with sbuild:

sbuild -A -d precise package.dsc

This will build the package on precise for ALL available architectures. Note that -d is just “precise”; the -A flag will tell sbuild to build architecture: any packages for all available architectures (so if you have amd64 and i386 chroots, it’ll do the right thing and build two packages).

If you want to build arch-specific packages:

sbuild  -d precise-i386 package.dsc

This will magically build for the given architecture (i386). Note that arch: any packages will also be built.

You can also specify the arch as a parameter (but then you have to leave it out of the -d name):

sbuild  -d precise --arch=i386 package.dsc

This will not work:

sbuild  -d precise-i386 --arch=i386 package.dsc

Using diff on the output of two commands – named pipe and bash magic

Categories: Uncategorized

Ever wanted to diff the output of two commands? Usually it’s done by first piping each command to a temporary file and then diffing them.

The following syntax creates a named pipe for the command and uses the pipe’s name instead of a filename. Bash takes care of everything automagically so all you have to do is:

sort <(cat /etc/passwd)

That’s a dumb example, but how about this?

diff <(command1) <(command2)

The commands can be as complicated as you need them to be!

Why I’m staying on Unity

Categories: English Geeky

A very interesting conversation erupted today, beginning when a coworker sent a lengthy email stating his reasons for altogether leaving Ubuntu 11.04’s new Unity desktop interface and instead resorting to the good, old-fashioned Gnome 2 “Classic” session.

In it he makes some very valid points about functionality that’s different to what he was used to. This understandably affects his workflow, so instead of wrestling with a new interface, he chose to go with the old one, hopefully until Unity matures enough for him to be able to customize it to his liking.

Unity bar

What’s interesting was the amount of responses it got, where everyone spoke about their “pet peeves” with Unity. The vast majority were changes in how Unity handles things, that interfered with people’s workflows. It’s understandable that even a small change in how your user interface behaves, when you’ve become adept at working with it, disrupts things enough (and annoyingly enough) that you either go back to the old user interface, or just start fiddling with the new one until you find a way to get things to an acceptable state.

Which is what struck me as curious about this thread: there were basically two camps, those who flat out abandoned Unity for the time being, and those who actually went looking into how Unity behaves and integrates with the environment, and came up with ways to make Unity more comfortable to those used to the “old ways” of Gnome 2.x and its desktop interface.

Gnome

Without demerit to the original poster, whose points were quite valid, a lot of responses suggested ways to solve about 80% of his complaints about Unity. However, the fact that it took a team of experts to solve the problems that a user (and another expert, at that) was experiencing, is testament to the fact that Unity could still be made more intuitive, easier and more customizable.

I finally upgraded to Ubuntu 11.04 and Unity this past weekend. Like many, I experienced some usability issues, where the desktop wasn’t behaving the way I was used to. However, my use of the system means that I basically want the UI to stay out of my way. So the main change I had to make was to get the Unity dock to auto-hide, so that it only appears when I ask it to. The rest of the time it’s hidden away. Everything else, well, it’s admittedly different than what I’m used to, but that’s change for you. Was Unity making a change for change’s sake? Maybe so, but I think it’s change in the right direction. Even if it somewhat alienates experienced users (for whom, however, workarounds exist that handle nearly all their concerns), I think the true success of Unity is in how it works for new users. And here are two examples.

Another coworker posted his experience with showing Ubuntu and Unity to a newbie, fresh-from-Windows user. The user’s comments were along the lines of “this looks nice”, “It’s easy to use” and “I’m keeping it”.

Also, even though some have complained about the app lens being hard to use (and it’s a complaint I’ve seen already twice), I’ve seen users realize “but hey, if it’s really that messy, you can use the search field to find what you need, right?”. So yes, end users are realizing this, and it’s just a matter of polishing how things work. If all, I think it’s great to move users away from the “the computer has only two buttons” mindset and get them using the keyboard a little more.

So yes indeed, I’m staying on Unity, and I’m looking forward to seeing it maturing into a better desktop interface. as Mark Shuttleworth said, it’s a foundation on which the next generations of Ubuntu user experience will be built. I’ll be thrilled to be along for the ride.

Finally, for a great write on why your desktop changed, and why the developers would appreciate you giving it a whirl and helping improve it (even just commenting on the stuff you find hard, unintuitive or just plain wrong) is better than just swearing off these newfangled changes (without which, face it, you’d still be using fwm and MIT Athena widgets), please drop by Federico Mena-Quintero’s activity log and read his wonderful and short article “Moving into your new Gnome 3 house“.