phly, boy, phly - Linux

Apache HOSTNAME on Clusters

nospam@example.com (Matthew Weier O'Phinney) — Fri, 25 Jan 2008 17:38:16 -0500

In an effort to debug issues on a cluster, I was trying to determine which machine on the cluster was causing the issue. My idea was that I could insert a header token identifying the server.

My first idea was to add the directive 'Header add X-Server-Ip "%{SERVER_ADDR}e" in my httpd.conf. However, due to the nature of our load balancer, Apache was somehow resolving this to the load balancer IP address on all machines of the cluster -- which was really, really not useful.

I finally stumbled on a good solution, however: you can set environment variables in apachectl, and then pass them into the Apache environment using the PassEnv directive from mod_env; once that's done, you can use the environment variable anywhere.

In my apachectl, I added the line "export HOSTNAME=`hostname`". Then, in my httpd.conf, I added first the line "PassEnv HOSTNAME", followed by the directive 'Header add X-Server-Name "%{HOSTNAME}e"'. Voila! I now had the hostname in the header, which gave me the information I needed for debugging.

Gutsy Gibbon review

nospam@example.com (Matthew Weier O'Phinney) — Sat, 20 Oct 2007 07:55:00 -0400

Early in the week, I decided to avoid the release rush and go ahead and update my laptop to Ubuntu's Gutsy Gibbon release. Overall, it's quite good, with one caveat I'll elaborate on later.

Continue reading "Gutsy Gibbon review"

Back on Linux Again

nospam@example.com (Matthew Weier O'Phinney) — Sat, 17 Feb 2007 13:50:58 -0500

A little over a year ago, I stopped using Linux as my primary desktop due to the fact that a number of programs we were using were Windows dependent. Despite getting coLinux running, I've never been completely satisfied with the setup. I missed being able to paste with my middle-mouse button, and I was constantly having character encoding issues pasting back and forth between PuTTY and windows apps, couldn't access mail easily between my coLinux and Windows partitions, and overall felt that I was losing out on some productivity by not having a native linux environment as my primary OS.

Last week, we had an infrastructure change at work, and I basically realized that my Windows + coLinux setup was going to get in the way of productivity -- and that, at this point, there were now Windows applications tying me to that OS. So, I decided it was time to go back to Linux.

Continue reading "Back on Linux Again"

coLinux Recovery

nospam@example.com (Matthew Weier O'Phinney) — Mon, 25 Sep 2006 17:27:00 -0400

As I've written previously, I use coLinux in order to have a Linux virtual machine running on my Windows XP install. It runs Debian unstable (SID), which gives me all apt-geet love I could want.

Except when an apt-get based install goes bad, that is, like it did Saturday evening. This is the tale of how I got it back up and running.

Continue reading "coLinux Recovery"

XP + Cygwin + coLinux == Productivity

nospam@example.com (Matthew Weier O'Phinney) — Thu, 05 Jan 2006 16:57:00 -0500

I wrote earlier of my experiences using Windows XP, a move I've considered somewhat unfortunate but necessary. I've added a couple more tools to my toolbox since that have made the environment even better.

Continue reading "XP + Cygwin + coLinux == Productivity"

File_SMBPasswd woes

nospam@example.com (Matthew Weier O'Phinney) — Wed, 07 Sep 2005 16:20:00 -0400

I've been cobbling together a system at work for the last couple months to allow a single place for changing all network passwords. This includes a variety of database sources, as well as passwd files and smbpasswd files. I've been making use of PEAR's File_Passwd and File_SMBPasswd, and they've greatly simplified the task of updating passwords for those types of systems. However, I've encountered some issues that I never would have expected.

I have the web user in a group called 'samba', and I have the smbpasswd file owned by root:samba. I then set the smbpasswd file to be group +rw. Simple, right? The web user should then be able to update the smbpasswd file without a problem, right? Wrong.

I kept getting errors, and on investigation continually found that the smbpasswd file permissions had reverted to 0600 -- i.e., only the root user could access it. I tried using 'chattr -i' on the off-chance that the file had been made immutable (which didn't make sense, as I was able to see the permissions change). No luck.

Based on observations of when the permissions reverted, it appears that the various SMB processes will reset the permissions! An example is when someone attempts to mount a resource from the server; this accesses the smbpasswd file to perform authentication -- and at this point the file permissions change. I can find no documentation to support this; these are simply my observations.

So, to get around the behaviour, I created a script that will set the file permissions to what I want them, and then gave sudo privileges to the samba group for that script. This script is then called via system() in the update script just before processing.

It's a hack, and could be made more secure, but it works.

Server upgrades, samba, and permissions, oh my!

nospam@example.com (Matthew Weier O'Phinney) — Fri, 19 Aug 2005 11:52:41 -0400

Last week, we performed a long needed upgrade to the company file/print/intranet server. Basically, we'd been on a Redhat 8 system, and there were upgrades we were wanting to perform, and couldn't due to lack of libraries. We could have possibly compiled from source in some occasions... but that would likely have shuttled us into a similar dependency hell as using Redhat in the first place.

So, we decided to re-install the OS, and switch to Gentoo in the process. We've found that Gentoo is a great distro for servers -- it allows us to tailor the install to the server purpose, and simultaneously provides a clean upgrade path via portage.

Things went primarily without a hitch. We lost a few databases due to a bad DB backup (argh! there went the wiki!), but that was the primary extent of the damage.

When investigating the sytem post-install, I discovered some connectivity issues with Samba. Basically, when connecting via a *nix-based machine, we were getting symlinks reported as being local to the connecting machine, not the server. This meant that symlinks on the server weren't being followed -- which caused major issues for those connecting via FTP, Mac, or Linux.

Continue reading "Server upgrades, samba, and permissions, oh my!"

PHP, Cgiapp, and extensibility

nospam@example.com (Matthew Weier O'Phinney) — Sat, 02 Apr 2005 21:41:10 -0500

At work this week, Rob was doing some monitoring of our bandwidth usage. We have SNMP on each of our servers now, and he uses MRTG to create bandwidth usage graphs that are updated every five minutes or so. He's been monitoring since late last year.

Before January, we had two systems going. The first, legacy, system hosted the majority of the content from garden.org, and was done using Tango 2000, a web application server that ran on top of IIS and Windows NT 4. I say 'ran', because Tango 2000 was the last version to ship; the company that made it stopped supporting it a year later. This meant we could not upgrade our server's OS to Windows 2000 or 2003, nor could we switch to a more secure web server, etc. It was a time bomb waiting to happen.

The second system is a basic LAMP system -- Linux + Apache + MySQL + PHP. Rob began migrating applications to it shortly after he started at NGA 3 years ago, one application at a time. Mostly, new applications were placed on it, though in May 2003, he and the other programmer who was there at the time started migrating old applications to the techology. Part of the reason I was hired was to continue this migration.

The migration was time consuming, and plenty of other projects often got in the way. However, starting last July, we made a big push to get it all ported over -- before the old WinNT server fails on us. In January, we were able to rollout the new garden.org, which runs on this new technology.

A big reason we were able to finish is because of Cgiapp. I originally ported it to PHP last year around this time, and knew that while I wanted to develop new applications using it, I wasn't so sure I could sell Rob on it.

Amazingly, it didn't take much to convince him. We had already started using Smarty for templates just before this, and were also using OOP in new development. Cgiapp just helped unify these technologies and to provide a nice, standard framework with which to program.

This last can not be emphasized enough. We started developing all applications in three places: an API for data access, a Cgiapp-based application, and our templates. Either one of us could pick up development of an application from the other without having to spend a day or two familiarizing ourselves with the idiosyncracies of what the other had decided was the programming paradigm of the day. Sure, we still have our own programming styles, but the framework makes it easy to debug or extend each others programs painlessly.

Now, back to the bandwidth reports: Rob has noticed that our bandwidth usage has been growing steadily on the new server since we switched garden.org over -- a 45 degree line. At one point this week, our outgoing bandwidth was almost 3 T1s -- and we were having no performance issues whatsoever. This simply would not have been possible on the old system -- nor without Cgiapp. We've managed to produce both a hardware architecture and a programming framework that has proved immensely scalable -- which will in turn save the organization money.

I love open source! How else can you create such high-performing software without paying through the nose for it?

Get Firefox!

nospam@example.com (Matthew Weier O'Phinney) — Fri, 24 Sep 2004 10:33:30 -0400

Those who know me know that I love linux and open source. One particular program that firmly committed me to open source software is the Mozilla project -- a project that took the Netscape browser's codebase and ran with it to places I know I never anticipated when I first heard of the project.

What do I like about Mozilla? Well, for starters, and most importantly, tabbed browsing changed the way I work. What is tabbed browsing? It's the ability to have multiple tabs in a browser window, allowing you to switch between web pages without needing to switch windows.

Mozilla came out with a standalone browser a number of months back called, first Phoenix, then Firebird, and now Firefox. This standalone browser has a conservative number of basic features, which allow for a lean download -- and yet, these basic features, which include tabbed browsing and disabling popups, far surpass Internet Explorer's features. And there are many extensions that you can download and integrate into the browser.

One such extension is a tabbed browsing extension that makes tabbed browsing even more useful. With it, I can choose to have any links leaving a site go to a new tab; or have bookmarks automatically load in a new tab; or group tabs and save them as bookmark folders; or drag a tab to a different location in the tabs (allowing easy grouping).

Frankly, there's few things I can find that Firefox can't do.

And, on top of that, it's not integrated into the operating system. So, if you're on Windows, that means if you use Firefox, you're less likely to end up with spyware and adware -- which often is downloaded and installed by special IE components just by visiting sites -- ruining your internet experience.

So, spread the word: Firefox is a speedy, featureful, SECURE alternative to Internet Explorer!

Get Firefox

What's keeping that device in use?

nospam@example.com (Matthew Weier O'Phinney) — Sat, 11 Sep 2004 21:27:11 -0400

Ever wonder what's keeping that device in use so you can't unmount it? It's happened to me a few times. The tool to discover this information? lsof.

Basically, you type something like: lsof /mnt/cdrom and it gives you a ps-style output detailing the PID and process of the processes that are using the cdrom. You can then go and manually stop those programs, or kill them yourself.

Binary packages for Gentoo

nospam@example.com (Matthew Weier O'Phinney) — Tue, 27 Apr 2004 22:48:53 -0400

I'd read that you could get binary packages for gentoo, thus alleviating the need to compile everything. (Of course, then you lose some of the benefits of compiling everything, but you gain in speed...) Unfortunately, I mistook this with ebuilds, and never quite figured it out.

The key is to throw the -g flag:

    % emerge -g gnumeric # which is like 'emerge --getbinpkg gnumeric'

I also learned how to update packages tonight:

    % emerge sync             # to sync your mirror with the gentoo mirrors
    % emerge --update portage # if necessary
    % emerge --update system  # updates core system files
    % emerge --update world   # updates all packages

Gentoo for several weeks

nospam@example.com (Matthew Weier O'Phinney) — Thu, 22 Apr 2004 22:10:10 -0400

I've had a bunch of problems with my new computer -- it uses ACPI, but if I load the ACPI modules, it won't boot; if I don't load them, I have to go through contortions to get the ethernet working, and it won't power down; and a bunch of other little stuff.

So, a few weeks ago, I thought, what the heck? Why not try Gentoo? I've been reading about it since it first came out, and I remember talking with Duane about it once, and it has a reputation for both being cutting edge and stable. Heck, even Wil Wheaton's endorsing it... it can't be all bad, right?

I had a few misstarts -- bad CDs, not quite getting how the chroot thing worked, problems with DNS (which I still don't understand; and Rob has them as well, so it's not just me). But once I got it installed... well, I'm impressed.

The thing about Gentoo is, it compiles everything from source. It's like Debian, in that it fetches all dependencies and installs those... but it's all source. So it's not exactly fast. But because everything is compiled, and because you setup C flags specific to your machine, what you get is incredibly optimized for your own machine. This 1.6GHz machine simply flies. And the memory usage just stays low.

I'd like to use it for my server... but I can't really take the server down at this point when it's making both my mom and myself money. But what a great system... I only wish I'd used it for the mail server at work.

B. Gates: Open Source Programmer?

nospam@example.com (Matthew Weier O'Phinney) — Mon, 29 Mar 2004 13:02:16 -0500

I just read coverage of a panel of programming luminaries on Salon; the topic of discussion was about the state of programming. In the course of the discussion, the subject of Open Source came up. Several of the luminaries -- which included architects of the Mac OS and Windows, as well as others -- derided the community for basically reinventing the wheel, and wheels that need to be re-thought entirely anyways. One questioned, "Why is hte idealism just about how the code is shared -- what about idealism about the code itself?"

Andy Hertzfeld (who helped develop the original Mac OS) was sitting on the panel, and jumped in. He has been working with Eazel and Chandler in recent years, and thus has an inside view of open source. His initial comment: "It's because they want people to use the stuff!" Basically, they program Windows- or Mac-like interfaces because then people will be willing to try it out. They program office suites because people "need" an office suite to be productive. Such offerings hook them into the OSS movement.

Another participant, Dan Bricklin (of VisiCalc, a pioneering spreadsheet program) shared an anecdote from Bill Gates. Evidently, Gates gave an interview (with Lammers -- look up this person) in which he explained that his work on MS's BASIC compiler was done by looking at how other programmers had accomplished the task. In his own words, "The best way to prepare is to write programs, and to study great programs that other people have written. In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating systems."

So basically, Gates was an early adopter of OSS methodologies... Interesting to see that today he's so protective of MS code. Guess money might do that to you.

Making a RAID array from the command line

nospam@example.com (Matthew Weier O'Phinney) — Thu, 11 Mar 2004 21:34:18 -0500

Last night, I created my first RAID array from the commandline. It was quite simple, I discovered.

Create your partitions using fstab. Remember, primary partitions must be created before extended partitions.
Look in /proc/partions and note the new partition IDs.
Edit /etc/raidtab and create a new RAID array. If unsure of the syntax, look up the Linux Software RAID HOWTO for more details.
Type mkraid /dev/md?, where ? is the id of the RAID device you just entered in /etc/raidtab.
Format the new RAID device with your favorite filesystem, assign it a mount point, and start using it!

I was impressed with how easy it was; the choices that the Anaconda installer present for creating a RAID array made it seem like the underlying process must be difficult, when in fact it may have been almost the same complexity if not easier.

Learn something new everyday

nospam@example.com (Matthew Weier O'Phinney) — Mon, 16 Feb 2004 07:33:39 -0500

Linux.com has had a running series on CLI commands for Newbies. Most of it has been very basic, but there are still a few gems within. For instance, today I was introduced to apropos and whatis. Give a search term to the former, and it will list all programs in which the search term is found in the manpages; give a program name to the latter, and it will tell you which man page addresses it.