Filed Under (Geekstuff, Linux, My Music) by Sean on March-3-2008

This is a follow up to the previous post located here.

Well I finally decided to get down and dirty once again and dive into the innards of real time kernel tuning. As I reported last time, I was getting some latency which was evident through the sound card as a very nasty buzzing sound. Here I’ll report some basic findings which hopefully will help anyone attempting to set up a real time system set up for use as a DAW.

My system is running Ubuntu Gutsy Gibbon. You may recall where I had some serious trouble installing the Ubuntu Studio real time Ubuntu kernel; the initrd image was severely anemic to the point where I couldn’t even boot up. Instead, I decided to download the latest kernel source and roll my own so I knew everything would be just the way I wanted.

Before I go into the step by step process of how that was done, allow me to report some updated findings that may clear up some confusion. One issue is deciding what method to use to grant realtime scheduling to users (so you don’t have to run programs as root, which is a very bad idea). The old way of doing things was to use the realtime-lsm kernel patch and module. The problem I was running into with this method along with the latest 2.6.24 kernels is the fact that you can no longer configure your kernel with CONFIG_SECURITY_CAPABILITIES as a module! The kernel maintainers have decided that it introduces some serious security vulnerabilities, so the only options now are built into the kernel or not at all. This is a serious problem when using realtime-lsm since presumably the capability module does not support stacking and needs to be unloaded when the realtime-lsm module is loaded. This is not completely true as pointed out by the Debian Kernel Team, but the realtime-lsm module patches I was using have not been modified to correct this. In any case, realtime-lsm has been deprecated in favor of the newer PAM rlimits method of granting realtime scheduling priority to userspace programs. It is important to note that realtime-lsm will still give you superior performance over the rlimits method for really heavy multimedia work, but in 99% of cases the rlimits method should suit most users just fine. The rlimits method works by configuring priority scheduling limits in /etc/security/limits.conf.

Let’s get to the meat of how I configured my system. I’m going to list all the steps in order with a very brief explanation of each.

First we get the latest kernel source tarball. In my case this was 2.6.24.3.

wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.xx.tar.bz2

Then we get the latest realtime kernel patch. Make sure the patch version matches your kernel version above. Get the latest rt version if there is more than one for your kernel. In my case this was patch-2.6.24.3-rt3.bz2.

wget http://kernel.org/pub/linux/kernel/projects/rt/patch-2.6.xx-rt.bz2

Copy the files to your kernel source tree, un-tarball them and patch the kernel. Be sure to use the proper version numbers below in place of these as applicable from your download.

cd /usr/src/

tar zjfv ~/linux-2.6.24.3.tar.bz2 linux-2.6.24.3 .

ln -s linux-2.6.24.3 linux

cd linux

bzcat ~/patch-2.6.24.3-rt3.bz2 | patch -p1

Get your current working config if you have one.

zcat /proc/config.gz > .config

or

cp /usr/src/linux-`uname -r`/.config ./.config

From the existing .config, if you have one, update the settings from it. If there are any new features from the last config file, you may be asked some configuration questions.

make oldconfig

Get your favorite editor out and modify the config file to match the following settings. If they don’t exist just add them to the end of the config file. Another option is to make menuconfig (or xconfig, gconfig, config, etc.) and set the values that way.

vi .config

HIGH_RES_TIMERS=y

PREEMPT_RT=y

HZ_1000=y

RTC=y

SND_RTCTIMER=y

SND_SEQ_RTCTIMER_DEFAULT=y

Now we can recompile and install our kernel. Below is the Debian/Ubuntu way of doing it. You can use the old school make && make modules && make modules_install technique as well. The Debian way does it all for you and packages the kernel image and headers as .deb packages in /usr/src and even installs the new kernel into GRUB!

make-kpkg --initrd --append-to-version=-rt_custom kernel_image kernel_headers

dpkg -i ../*deb

Now we grant realtime scheduling to the user group audio. Be sure your user account is part of this group. Here we set a max priority of 99 (RT), nice value of -10 (not very nice :)) and memlock value of 512MB.
sudo cat "@audio - rtprio 99" >> /etc/security/limits.conf
sudo cat "@audio - nice -10" >> /etc/security/limits.conf
sudo cat "@audio - memlock 512000" >> /etc/security/limits.conf

Verify everything looks good in GRUB and reboot. Verify you’re running the new kernel. Look for PREEMPT RT in the version.

uname -ar

Linux ubuntu 2.6.24.3-rt3-rt_custom #1 SMP PREEMPT RT Sun Mar 2 22:06:11 CST 2008 i686 GNU/Linux

Great! Now you can run qtjackctl and select the appropriate realtime setting. You can launch your other studio applications with these settings as well and modify them in real time using the chrt command. You can check on the status of a process with ps -eo rtprio,comm.

ps -eo rtprio,comm | grep jack

chrt -p 99 <process id>

I will follow up with some more details and howtos regarding working with your DAW. Feel free to comment below if you have any questions or email me using the email icon at the top of the page. Enjoy!



Filed Under (Geekstuff, Rants, Technology) by Sean on February-28-2008

This is a very minor update to my series of rants about Microsoft’s crappy software.

So as mentioned in the last rant, I was backing up all the data on my Dell laptop before I wipe it clean and install Ubuntu. I left the office yesterday after starting the backup process. I came in today expecting it to be complete. Nope! Guess what? Yep, Microsoft decided it was a better idea to reboot my machine after some very important updates instead of complete my backup. It only backed up 1.7 GB, so now I’ve got to start again . . . . sheesh . . .



Filed Under (Geekstuff, My Music) by Sean on February-19-2008

Wow . . . I certainly didn’t anticipate the difficulty I was going to face trying to get all the software set up and configured on my shiny new Digital Audio Workstation.

You see, it’s all about real time computing. Since normal, every day Linux is based on a voluntary preemptive kernel (as are all other regular OS-es), it is not suited to the demands of a professional DAW. The various software components of the DAW have to sync perfectly and in real time. The sequencers, synthesizers, mastering software, loops, keyboards connected though MIDI and audio, all have to exactly sync up with near zero latency if you’re going to have anything that sounds worth a crap come out the other end.

UbuntuStudio was created with this in mind. It has the real time kernel included, all the DAW software and components that tie it all together (the zero latency plumbing is called Jack). I had already installed Ubuntu Desktop (Gutsy Gibbon to be precise) so I followed the instructions to upgrade. Well something definitely went awry with that process. For some reason, the install of the rt kernel totally futzed up my initrd image to the point where it didn’t detect my hard drive. Suffice it to say that makes it difficult to start mastering any recordings!

I first attempted the usual fixes, including yaird, initrd, initramfs tools, to no avail. I went back to my default kernel and went ahead and launched all my DAW software anyway, setting up Jack with options so it would ignore the latency problems. I just wanted to see how the applications worked in general. Everything ran, albeit not synced in real time.

So, I decided to roll my own kernel from scratch, pulling it from kernel.org, patching it with the realtime kernel patch, choosing the appropriate kernel options, and compiling. After a few false starts (didn’t quite get all the correct options in the menuconfig the first couple of times) and a few hours, I had a kernel that finally booted! Finally my optimized, preemptive realtime kernel:

$uname -ar
Linux sean-desktop-home 2.6.24-rt1-rtsbs #1 SMP PREEMPT RT Mon Feb 18 22:25:02 CST 2008 i686 GNU/Linux

Of course now that I was no longer using the restricted mode NVidia drivers to get hardware acceleration (i.e., the restricted module doesn’t match the kernel version; I’ll have to download and compile module from NVidia), I didn’t have a working X server. A simple dpkg-reconfigure xserver-xorg later and I was off and running. Click on the image below to see it in action.

My DAW on Ubuntu

I still was getting xruns on Jack, but my SBLive card has issues anyway. Sometimes the synthesis or even regular sound files sound like ass; it is a 4 year old card that cost $20 so I can’t complain. I guess I’m going to have to shell out for the E-mu 1212M I’ve been eying and get down to business. I suspect that will solve a lot of my problems. Who knows, I may even finally get some of my music uploaded as promised. Stay tuned for more updates (and more late nights) soon!



Filed Under (Geekstuff, My Music) by Sean on February-17-2008

So now that I’ve decided I’m going to get serious about releasing the music I’ve composed for the better part of the last 25 years (and still write), I’ve been going through the process of learning as much as I can about Digital Audio Workstations (DAW). I’ve certainly played around with ProTools and other DAW software over the years but I really wanted to move everything to 100% open source running under Linux (see my previous article about moving exclusively to Ubuntu on all my systems). It appears the time has come for that to become a reality. I’ll be writing here about my experience in getting my DAW set up so others will gain from it. Here is the very rough list of the components of my DAW:

I still need to purchase either a good full-size electric piano or steal my mother’s Steinway Grand I had the privilege playing growing up. :) Another idea is to get a good, 88-key, weighted midi controller and use sampled sounds. I have written and arranged lots of multi-timbred works but much of my continuing composition these days is solo piano.

I will keep you posted on my continuing efforts. Also stay tuned for some initial recordings coming very soon!

(UPDATE) See http://www.seanstoner.com/blog/2008/02/19/linux-studio-chapter-ii/ for the follow up to this post.



Filed Under (Business, Internet) by Sean on February-4-2008

I was certainly as surprised as anyone to see Microsoft’s $46 Billion bid for Yahoo. In my opinion this prospective acquisition ranks somewhere between the most ill-conceived idea ever to a decidedly mediocre, ho-hum idea.

It seems at first pass this is a great deal to short term Yahoo shareholders who are looking for an exit. Yahoo’s stock and innovation alike have languished for years. I believe Yahoo’s woes result from an identity crisis. Are they a media company? A technology company? An Internet search engine? A software company? A software as a service (SAAS) company? Who the hell is Yahoo? As a result Yahoo has, in a way, suffered double jeopardy from this lack of identity. They have aligned their market capitalization to that of traditional media companies, while completely stalling innovation as a technology company.

I think Microsoft and Yahoo have it completely wrong — “it” being strategy. Google has it right, pure and simple. The race is a race of innovation. Google’s innovation has continued to accelerate since its founding. Microsoft continues to dump its decade old software into a blender and spit out boring variations on the same theme. It uses its monopolistic hegemony, ties with PC manufacturers and extortion over its existing install base to keep market share. Google has innovated new and exciting ideas in software to get things done, which has resulted in its meteoric rise. Yahoo once had this innovation bug too, until it decided it wanted to become a media outlet sometime around 2001; it has paid the price ever since.

I believe I can speak with some authority on these three companies since I’ve been a paying enterprise customer of each (in the tens of thousands of dollars and beyond); presumably I’ve seen the best they have to offer and have counted on them to provide mission critical services. In the last 6 months, I have entirely moved away from both Yahoo and Microsoft as a paying customer and user — a timely and fortuitous move in my opinion. I have moved away from both for what can be summed up thusly: please stop telling me how I’m supposed to do things. It seems whenever I try to accomplish a task, their software dictates how I’m supposed to do it rather than the other way around. I think it’s ironic that, as a paying customer of Microsoft and Yahoo, I decided virtually simultaneously to ditch both mere weeks before this development. I don’t think I’m the only one doing so, either. Does Microsoft think that by combining with Yahoo they can stall or reverse the flight? I think not.

So what do I use these days to get things done? I switched to Open Office (which also forms the back-end of Google Docs) to do all of my basic document creation and Ubuntu Linux for my desktop computer’s operating system. I use a mashup of Google Calendar, Contacts, Evolution, my Treo and some web services middleware to keep it all in sync for my PIM. I have been using Linux as a server platform and built an entire company on it since its beginnings in the early 1990s. Back then, when I wanted to implement a feature in software that wasn’t available (which was often it seemed), I would simply dive into the source code, add the features I needed and recompile with my trusty C compiler. Try doing that with Microsoft software! In some cases, where I still have to use Internet Explorer to visit websites designed by Neanderthals that still require IE instead of working with standards compliant browsers like Firefox (I find it amusing that Microsoft’s own web site’s simple “About Us” page has roughly 195 validation errors), I use Wine, which works amazingly well. I also use numerous other open source, free applications from repositores containing thousands of increasingly mature applications. I will devote an entire post to typical applications used for various tasks as well as open source alternatives/equivalents to brand name software (hint: want to install a Photoshop-like graphics application NOW? type “sudo apt-get install gimp” or “yum install gimp” and presto! It’s installed and ready to use within seconds. That was easy!) My apologies, however, as I digress. I do not intend to foment the proprietary vs. open source debate here; that will be another post entirely.

Let’s discuss Yahoo’s languishing innovation, for example. I’ve used their Mail Plus web application for email for many years. One of the things we all deal with is spam. No matter how good spam filtering software is, even the most sophisticated, trained Bayesian filters trap false positives. Therefore it is important to cull through your spam folder to mark these emails as ham (i.e., not spam) to train your spam filter to not trap them in the future. The easiest way I know to do this is perform simple searches of your spam folder and look for legitimate senders and the like. My spam folder collects over 5,000 messages each month. What is my problem? Yahoo doesn’t let you search your spam folder. With the number of users commenting on this oversight in Yahoo forums, you would think Yahoo would listen and correct this problem post haste. In fact, I emailed Yahoo customer support about this issue more than 4 years ago. They still haven’t done anything about it. Presumably, from a software perspective, the reasoning for this is that spam outnumbers legitimate email 6 to 1 and indexing this would add an unnecessary burden on Yahoo’s compute infrastructure. But they are (were) a search company, right? They give you a work-around but it is virtually impossible to implement with anything more than a handful of messages. Google let’s you search your spam mail just like normal email, as you would expect. Google, quite frankly, just “gets it.” Yahoo and Microsoft don’t.

I could bore you with numerous other examples exemplifying huge holes in Yahoo’s business, especially when it comes to their horribly contrived customer support system (even after a perfectly composed, detailed email describing an obvious bug including all relevant details fit for a quality assurance team, you get a ridiculous response prompting you to perform insipid tasks such as checking to see if your computer is plugged in and the like), but I’ll get back to the thesis of this article.

Microsoft is another lost company. I have a great deal of respect for Bill Gates, but I think Steve Ballmer seriously needs to cut down on LSD or whatever deleterious hallucinogens he may be using. Their problems are compounded by pathology within MS that their shit doesn’t stink. In my opinion MS would benefit greatly by adopting open source and creating value on top of it in areas where they excel such as user interface design and features (though that’s arguable with the latest version of Office as an example). Vista was an absolute waste of millions of development man-hours that could have been better used bettering the best-OS-MS-has-ever-made Windows 2003 line up (XP and 2003 Server). I mean, seriously! One of the big deals about Windows NT was its goal to be fully POSIX compliant. Seems they completely forgot what they were trying to do. Even now, 15 years after NT’s release, Microsoft operating systems (still based on the NT kernel architecture) still lack probably the single most powerful aspect of POSIX and *nix systems - a powerful, scriptable, robust command line shell - a must for any serious server administrator’s tool chest.

All in all, the reasons a Yahoo acquisition is a bad deal are many. For Yahoo, it would mean the demise of a company culture that could once again foster great innovation. It would mean some promising software products, including recent acquisitions, could hit the chopping block (Zimbra, Flickr). It would be bad for Yahoo customers who went to Yahoo to get away from Microsoft in the first place. It would mean melding two distinctly different infrastructures (Yahoo is primarily based on open source software fundamentals and linux based platforms). It is bad for Microsoft for the same reasons (i.e., once the above things are priced into Yahoo stock after an acquisition, much of the value would be lost). From a strictly economic and shareholder return perspective, Microsoft would do better to buy back its own stock than acquire Yahoo.

Both companies are suffering from bad strategy. Microsoft is the blood-thirsty bully, reacting to competitive threats with attack posturing and hegemony rather than quietly winning the war through innovation, ala Google. However Microsoft does have a very talented workforce and very smart people which it could leverage if it weren’t for the raving lunatics in the executive suite and the prevailing proprietary technology dogma that should be laid to rest. Microsoft should launch into open source, deploy hostable, SAAS versions all its leading software, port its software to run on Linux and even create its own Linux distribution. Microsoft made a huge bet that the likes of John Gage were wrong with his infamous phrase, “The Network is the Computer.” Microsoft bet wrong. The war is on applications and functionality and efficacy of those applications to get things done. Not operating systems. Increasingly the Internet is the deployment medium for these applications. I suppose perhaps Microsoft has figured this out which is what the Yahoo bid is all about. I still believe, however, that just like Microsoft’s huge miscalculation about the importance of the Internet back in the 90’s, they are still running in circles based on flawed assumptions and fundamentals.

Similarly, Yahoo has the people, the smarts, the culture, market share and the infrastructure to really compete against Google. Its problem is simply lack of a coherent vision of what it is, what it is trying to do and why (and what it wants to be when it grows up). It does have a new mission statement, which is a good thing. Now they need to really put words into action. Microsoft is only buying Yahoo because of its brand and its existing audience, not because of its capability to innovate. I don’t believe mergers just to acquire customers ever work, especially when a large percentage of the customers you’re acquiring got there to begin with by avoiding you. I’m also speaking from personal experience; my Internet company in the 90’s was acquired by a company in an effort to acquire customers and not to continue our proven strategy of success (that will be another whole blog post). The problem was that their going-forward strategy was flawed and as a result the whole house of cards fell down.

In summary, I think this proposed deal is a bad deal on just about every front. Many news outlets and bloggers are calling the bid hostile but that remains to be seen. I only see it as unsolicited, but not yet hostile. Yahoo does have poison pill protection in the event they also see the takeover as hostile. It will be interesting to watch from the sidelines and see how things develop.