This is a follow up to the previous post located here.

Well I finally decided to get down and dirty once again and dive into the innards of real time kernel tuning. As I reported last time, I was getting some latency which was evident through the sound card as a very nasty buzzing sound. Here I’ll report some basic findings which hopefully will help anyone attempting to set up a real time system set up for use as a DAW.

My system is running Ubuntu Gutsy Gibbon. You may recall where I had some serious trouble installing the Ubuntu Studio real time Ubuntu kernel; the initrd image was severely anemic to the point where I couldn’t even boot up. Instead, I decided to download the latest kernel source and roll my own so I knew everything would be just the way I wanted.

Before I go into the step by step process of how that was done, allow me to report some updated findings that may clear up some confusion. One issue is deciding what method to use to grant realtime scheduling to users (so you don’t have to run programs as root, which is a very bad idea). The old way of doing things was to use the realtime-lsm kernel patch and module. The problem I was running into with this method along with the latest 2.6.24 kernels is the fact that you can no longer configure your kernel with CONFIG_SECURITY_CAPABILITIES as a module! The kernel maintainers have decided that it introduces some serious security vulnerabilities, so the only options now are built into the kernel or not at all. This is a serious problem when using realtime-lsm since presumably the capability module does not support stacking and needs to be unloaded when the realtime-lsm module is loaded. This is not completely true as pointed out by the Debian Kernel Team, but the realtime-lsm module patches I was using have not been modified to correct this. In any case, realtime-lsm has been deprecated in favor of the newer PAM rlimits method of granting realtime scheduling priority to userspace programs. It is important to note that realtime-lsm will still give you superior performance over the rlimits method for really heavy multimedia work, but in 99% of cases the rlimits method should suit most users just fine. The rlimits method works by configuring priority scheduling limits in /etc/security/limits.conf.

Let’s get to the meat of how I configured my system. I’m going to list all the steps in order with a very brief explanation of each.

First we get the latest kernel source tarball. In my case this was 2.6.24.3.

wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.xx.tar.bz2

Then we get the latest realtime kernel patch. Make sure the patch version matches your kernel version above. Get the latest rt version if there is more than one for your kernel. In my case this was patch-2.6.24.3-rt3.bz2.

wget http://kernel.org/pub/linux/kernel/projects/rt/patch-2.6.xx-rt.bz2

Copy the files to your kernel source tree, un-tarball them and patch the kernel. Be sure to use the proper version numbers below in place of these as applicable from your download.

cd /usr/src/

bzcat ~/patch-2.6.24.3-rt3.bz2 | patch -p1

Get your current working config if you have one.

zcat /proc/config.gz > .config

or

cp /usr/src/linux-`uname -r`/.config ./.config

From the existing .config, if you have one, update the settings from it. If there are any new features from the last config file, you may be asked some configuration questions.

make oldconfig

Get your favorite editor out and modify the config file to match the following settings. If they don’t exist just add them to the end of the config file. Another option is to make menuconfig (or xconfig, gconfig, config, etc.) and set the values that way.

vi .config

SND_SEQ_RTCTIMER_DEFAULT=y

Now we can recompile and install our kernel. Below is the Debian/Ubuntu way of doing it. You can use the old school make && make modules && make modules_install technique as well. The Debian way does it all for you and packages the kernel image and headers as .deb packages in /usr/src and even installs the new kernel into GRUB!

make-kpkg --initrd --append-to-version=-rt_custom kernel_image kernel_headers

dpkg -i ../*deb

Now we grant realtime scheduling to the user group audio. Be sure your user account is part of this group. Here we set a max priority of 99 (RT), nice value of -10 (not very nice :)) and memlock value of 512MB.
sudo cat "@audio - rtprio 99" >> /etc/security/limits.conf
sudo cat "@audio - nice -10" >> /etc/security/limits.conf
sudo cat "@audio - memlock 512000" >> /etc/security/limits.conf
Verify everything looks good in GRUB and reboot. Verify you’re running the new kernel. Look for PREEMPT RT in the version.

uname -ar

Linux ubuntu 2.6.24.3-rt3-rt_custom #1 SMP PREEMPT RT Sun Mar 2 22:06:11 CST 2008 i686 GNU/Linux

Great! Now you can run qtjackctl and select the appropriate realtime setting. You can launch your other studio applications with these settings as well and modify them in real time using the chrt command. You can check on the status of a process with ps -eo rtprio,comm.

ps -eo rtprio,comm | grep jack

chrt -p 99 <process id>

I will follow up with some more details and howtos regarding working with your DAW. Feel free to comment below if you have any questions or email me using the email icon at the top of the page. Enjoy!

No, I’m not talking about the strings from theoretical physics here, just plain old strings in computer programming.

So, let’s say you need to generate a truly random string for a password, encryption key salt, or some other purpose. For this purpose the best thing is to use the /dev/random or /dev/urandom device on your computer. If your system doesn’t have a hardware random number generator (e.g., palm devices, phones, embedded systems), you can use alternative techniques such as using Lava Lamps, WiFi background noise, Space or live Keno results from Vegas to generate truly random data.

Computers cannot, by themselves, generate truly random data. Most random algorithms are based on pseudo-random number generation, which are basically data that appear random but are actually generated from a completely deterministic process, making them not random at all by definition. The /dev/random device, which first appeared in Linux, uses environmental noise to generate its entropy pool from which random data can be created. Environmental noise includes keyboard and mouse use, device driver interrupt timing, and other non-deterministic data. The generator also stores the number of bits of noise that exist within the entropy pool. Thus when accessing /dev/random, if there isn’t sufficient entropy, the device will block until entropy is available. For example, if you call for a specific length string, and there isn’t sufficient entropy, it will generate what it can and wait for more entropy. You can generate entropy by using your computer’s mouse and keyboard (or do what I do and write a blog post or send out a tweet on twitter). If you cannot wait for the entropy pool to fill back up, the /dev/urandom device is available which is non-blocking since it reuses bits from the entropy pool to generate pseudo random bits (however sufficiently random for cryptographic use).

How do we create a random string then? Let’s say you need a 12 character string consisting of alpha numeric characters. This is the command I would use at a shell prompt (only type bold characters):

linux$ od -a -A n /dev/random | head -1 | tr -d ‘ ‘ | tr -d ‘\n’ | sed ’s/[^a-zA-Z0-9]//g’ | awk ‘{print substr($0,1,12)}’
stxeG5GyIack

You can replace /dev/random with /dev/urandom if you don’t need truly random data. If you need a longer string, you should increase the number used with the head command and modify the number in the substring (substr) function. Additionally, you can include symbols and other non-alphanumeric characters by eliminating the sed statement. If you need just numbers, you can use a different argument to od. Here are some examples of these in order:

linux$ od -a -A n /dev/random | head -30 | tr -d ‘ ‘ | tr -d ‘\n’ | sed ’s/[^a-zA-Z0-9]//g’ | awk ‘{print substr($0,1,256)}’
DhtBMcsiusdeltackeotbssyn7nakUsYhNuZnOhtspdVescdc4Xsili2CLkeoth8subnuleothtDackw
Sdc4bel2canSackdleSUsietb8Kdc3ynuleenqdc2ijbetb1euCMoTIXzemVyUFxketb69sdc1uQffD
subvjHGrlZpdc2AsinaknulSUrs7dc2nulnul4Gusus3vsp1GB025Kx3dc3xBddc40dleetxhrdc1yH
zu2usjffXMbswhtcaN


linux$ od -a -A n /dev/random | head -1 | tr -d ‘ ‘ | tr -d ‘\n’ | awk ‘{print substr($0,1,12)}’
J+O\amescIen

linux$ od -d -A n /dev/random | head -1 | tr -d ‘ ‘ | tr -d ‘\n’ | awk ‘{print substr($0,1,12)}’
140396406633

I hope you find this information informative. By typing this post, I’ve succeeded in filling back up my entropy pool!

Seriously, get a life, SCO and SNCP. In case you haven’t heard, SCO filed chapter 11 bankruptcy protection last year as a result of its failed lawsuits against Linux vendors and customers. Now it has courted a new suitor, Stephen Norris Capital Partners, to the tune of $100 million to continue it’s nonsensical war against open source and Linux.

At least companies that put out crappy software and/or fall seriously short in the ethics department still actually try to innovate and compete (e.g., Microsoft - though conspiracy theories abound that say they’re behind all this nonsense). This cause, however, is only about greed, opportunism, desperation, and psychopathy, pure and simple. Perhaps these guys ought to try to better life on the planet with their “largesse.” Perhaps someone could actually innovate with that money. You know, find a cure for cancer, or develop social programs to eliminate poverty. I certainly hope the bankruptcy court sees this for the extortionist fraud it is and severs it off at the head.

Even if their case has some merit (which it certainly doesn’t appear to given what’s transpired and SCO’s own tactics of declaring bankruptcy weeks before going to trial), from a moral perspective what they’re doing is simply wrong. With all that’s at stake, with users and governments around the world benefiting from the spread of free software, this is analogous to a rogue enterprise waging a trillion dollar intellectual property battle to blot out the use of antibiotics. It doesn’t make economic sense, legal sense, or business sense, and smacks of moral turpitude.

Since I’ve moved completely away from the evil empire (I’ve felt I’ve already gained a few extra years from abandoning Outlook), I’ve been looking at various ways to synchronize my Treo, Google Calendar, Google Contacts, Task lists, etc., so they all agree with one another. I was using a daemon for this purpose which used web services to keep things together, but it wasn’t as full featured in the Treo department as I would have liked. I came across this post and think I’m going to give it a shot. It is very thorough and has pretty pictures. What more can you ask for?

Wow . . . I certainly didn’t anticipate the difficulty I was going to face trying to get all the software set up and configured on my shiny new Digital Audio Workstation.

You see, it’s all about real time computing. Since normal, every day Linux is based on a voluntary preemptive kernel (as are all other regular OS-es), it is not suited to the demands of a professional DAW. The various software components of the DAW have to sync perfectly and in real time. The sequencers, synthesizers, mastering software, loops, keyboards connected though MIDI and audio, all have to exactly sync up with near zero latency if you’re going to have anything that sounds worth a crap come out the other end.

UbuntuStudio was created with this in mind. It has the real time kernel included, all the DAW software and components that tie it all together (the zero latency plumbing is called Jack). I had already installed Ubuntu Desktop (Gutsy Gibbon to be precise) so I followed the instructions to upgrade. Well something definitely went awry with that process. For some reason, the install of the rt kernel totally futzed up my initrd image to the point where it didn’t detect my hard drive. Suffice it to say that makes it difficult to start mastering any recordings!

I first attempted the usual fixes, including yaird, initrd, initramfs tools, to no avail. I went back to my default kernel and went ahead and launched all my DAW software anyway, setting up Jack with options so it would ignore the latency problems. I just wanted to see how the applications worked in general. Everything ran, albeit not synced in real time.

So, I decided to roll my own kernel from scratch, pulling it from kernel.org, patching it with the realtime kernel patch, choosing the appropriate kernel options, and compiling. After a few false starts (didn’t quite get all the correct options in the menuconfig the first couple of times) and a few hours, I had a kernel that finally booted! Finally my optimized, preemptive realtime kernel:

$uname -ar
Linux sean-desktop-home 2.6.24-rt1-rtsbs #1 SMP PREEMPT RT Mon Feb 18 22:25:02 CST 2008 i686 GNU/Linux

Of course now that I was no longer using the restricted mode NVidia drivers to get hardware acceleration (i.e., the restricted module doesn’t match the kernel version; I’ll have to download and compile module from NVidia), I didn’t have a working X server. A simple dpkg-reconfigure xserver-xorg later and I was off and running. Click on the image below to see it in action.

I still was getting xruns on Jack, but my SBLive card has issues anyway. Sometimes the synthesis or even regular sound files sound like ass; it is a 4 year old card that cost $20 so I can’t complain. I guess I’m going to have to shell out for the E-mu 1212M I’ve been eying and get down to business. I suspect that will solve a lot of my problems. Who knows, I may even finally get some of my music uploaded as promised. Stay tuned for more updates (and more late nights) soon!

I was certainly as surprised as anyone to see Microsoft’s $46 Billion bid for Yahoo. In my opinion this prospective acquisition ranks somewhere between the most ill-conceived idea ever to a decidedly mediocre, ho-hum idea.

It seems at first pass this is a great deal to short term Yahoo shareholders who are looking for an exit. Yahoo’s stock and innovation alike have languished for years. I believe Yahoo’s woes result from an identity crisis. Are they a media company? A technology company? An Internet search engine? A software company? A software as a service (SAAS) company? Who the hell is Yahoo? As a result Yahoo has, in a way, suffered double jeopardy from this lack of identity. They have aligned their market capitalization to that of traditional media companies, while completely stalling innovation as a technology company.

I think Microsoft and Yahoo have it completely wrong — “it” being strategy. Google has it right, pure and simple. The race is a race of innovation. Google’s innovation has continued to accelerate since its founding. Microsoft continues to dump its decade old software into a blender and spit out boring variations on the same theme. It uses its monopolistic hegemony, ties with PC manufacturers and extortion over its existing install base to keep market share. Google has innovated new and exciting ideas in software to get things done, which has resulted in its meteoric rise. Yahoo once had this innovation bug too, until it decided it wanted to become a media outlet sometime around 2001; it has paid the price ever since.

I believe I can speak with some authority on these three companies since I’ve been a paying enterprise customer of each (in the tens of thousands of dollars and beyond); presumably I’ve seen the best they have to offer and have counted on them to provide mission critical services. In the last 6 months, I have entirely moved away from both Yahoo and Microsoft as a paying customer and user — a timely and fortuitous move in my opinion. I have moved away from both for what can be summed up thusly: please stop telling me how I’m supposed to do things. It seems whenever I try to accomplish a task, their software dictates how I’m supposed to do it rather than the other way around. I think it’s ironic that, as a paying customer of Microsoft and Yahoo, I decided virtually simultaneously to ditch both mere weeks before this development. I don’t think I’m the only one doing so, either. Does Microsoft think that by combining with Yahoo they can stall or reverse the flight? I think not.

So what do I use these days to get things done? I switched to Open Office (which also forms the back-end of Google Docs) to do all of my basic document creation and Ubuntu Linux for my desktop computer’s operating system. I use a mashup of Google Calendar, Contacts, Evolution, my Treo and some web services middleware to keep it all in sync for my PIM. I have been using Linux as a server platform and built an entire company on it since its beginnings in the early 1990s. Back then, when I wanted to implement a feature in software that wasn’t available (which was often it seemed), I would simply dive into the source code, add the features I needed and recompile with my trusty C compiler. Try doing that with Microsoft software! In some cases, where I still have to use Internet Explorer to visit websites designed by Neanderthals that still require IE instead of working with standards compliant browsers like Firefox (I find it amusing that Microsoft’s own web site’s simple “About Us” page has roughly 195 validation errors), I use Wine, which works amazingly well. I also use numerous other open source, free applications from repositores containing thousands of increasingly mature applications. I will devote an entire post to typical applications used for various tasks as well as open source alternatives/equivalents to brand name software (hint: want to install a Photoshop-like graphics application NOW? type “sudo apt-get install gimp” or “yum install gimp” and presto! It’s installed and ready to use within seconds. That was easy!) My apologies, however, as I digress. I do not intend to foment the proprietary vs. open source debate here; that will be another post entirely.

Let’s discuss Yahoo’s languishing innovation, for example. I’ve used their Mail Plus web application for email for many years. One of the things we all deal with is spam. No matter how good spam filtering software is, even the most sophisticated, trained Bayesian filters trap false positives. Therefore it is important to cull through your spam folder to mark these emails as ham (i.e., not spam) to train your spam filter to not trap them in the future. The easiest way I know to do this is perform simple searches of your spam folder and look for legitimate senders and the like. My spam folder collects over 5,000 messages each month. What is my problem? Yahoo doesn’t let you search your spam folder. With the number of users commenting on this oversight in Yahoo forums, you would think Yahoo would listen and correct this problem post haste. In fact, I emailed Yahoo customer support about this issue more than 4 years ago. They still haven’t done anything about it. Presumably, from a software perspective, the reasoning for this is that spam outnumbers legitimate email 6 to 1 and indexing this would add an unnecessary burden on Yahoo’s compute infrastructure. But they are (were) a search company, right? They give you a work-around but it is virtually impossible to implement with anything more than a handful of messages. Google let’s you search your spam mail just like normal email, as you would expect. Google, quite frankly, just “gets it.” Yahoo and Microsoft don’t.

I could bore you with numerous other examples exemplifying huge holes in Yahoo’s business, especially when it comes to their horribly contrived customer support system (even after a perfectly composed, detailed email describing an obvious bug including all relevant details fit for a quality assurance team, you get a ridiculous response prompting you to perform insipid tasks such as checking to see if your computer is plugged in and the like), but I’ll get back to the thesis of this article.

Microsoft is another lost company. I have a great deal of respect for Bill Gates, but I think Steve Ballmer seriously needs to cut down on LSD or whatever deleterious hallucinogens he may be using. Their problems are compounded by pathology within MS that their shit doesn’t stink. In my opinion MS would benefit greatly by adopting open source and creating value on top of it in areas where they excel such as user interface design and features (though that’s arguable with the latest version of Office as an example). Vista was an absolute waste of millions of development man-hours that could have been better used bettering the best-OS-MS-has-ever-made Windows 2003 line up (XP and 2003 Server). I mean, seriously! One of the big deals about Windows NT was its goal to be fully POSIX compliant. Seems they completely forgot what they were trying to do. Even now, 15 years after NT’s release, Microsoft operating systems (still based on the NT kernel architecture) still lack probably the single most powerful aspect of POSIX and *nix systems - a powerful, scriptable, robust command line shell - a must for any serious server administrator’s tool chest.

All in all, the reasons a Yahoo acquisition is a bad deal are many. For Yahoo, it would mean the demise of a company culture that could once again foster great innovation. It would mean some promising software products, including recent acquisitions, could hit the chopping block (Zimbra, Flickr). It would be bad for Yahoo customers who went to Yahoo to get away from Microsoft in the first place. It would mean melding two distinctly different infrastructures (Yahoo is primarily based on open source software fundamentals and linux based platforms). It is bad for Microsoft for the same reasons (i.e., once the above things are priced into Yahoo stock after an acquisition, much of the value would be lost). From a strictly economic and shareholder return perspective, Microsoft would do better to buy back its own stock than acquire Yahoo.

Both companies are suffering from bad strategy. Microsoft is the blood-thirsty bully, reacting to competitive threats with attack posturing and hegemony rather than quietly winning the war through innovation, ala Google. However Microsoft does have a very talented workforce and very smart people which it could leverage if it weren’t for the raving lunatics in the executive suite and the prevailing proprietary technology dogma that should be laid to rest. Microsoft should launch into open source, deploy hostable, SAAS versions all its leading software, port its software to run on Linux and even create its own Linux distribution. Microsoft made a huge bet that the likes of John Gage were wrong with his infamous phrase, “The Network is the Computer.” Microsoft bet wrong. The war is on applications and functionality and efficacy of those applications to get things done. Not operating systems. Increasingly the Internet is the deployment medium for these applications. I suppose perhaps Microsoft has figured this out which is what the Yahoo bid is all about. I still believe, however, that just like Microsoft’s huge miscalculation about the importance of the Internet back in the 90’s, they are still running in circles based on flawed assumptions and fundamentals.

Similarly, Yahoo has the people, the smarts, the culture, market share and the infrastructure to really compete against Google. Its problem is simply lack of a coherent vision of what it is, what it is trying to do and why (and what it wants to be when it grows up). It does have a new mission statement, which is a good thing. Now they need to really put words into action. Microsoft is only buying Yahoo because of its brand and its existing audience, not because of its capability to innovate. I don’t believe mergers just to acquire customers ever work, especially when a large percentage of the customers you’re acquiring got there to begin with by avoiding you. I’m also speaking from personal experience; my Internet company in the 90’s was acquired by a company in an effort to acquire customers and not to continue our proven strategy of success (that will be another whole blog post). The problem was that their going-forward strategy was flawed and as a result the whole house of cards fell down.

In summary, I think this proposed deal is a bad deal on just about every front. Many news outlets and bloggers are calling the bid hostile but that remains to be seen. I only see it as unsolicited, but not yet hostile. Yahoo does have poison pill protection in the event they also see the takeover as hostile. It will be interesting to watch from the sidelines and see how things develop.