LinuxCon 2011: Day 1

Today was the first day of LinuxCon North America 2011. I managed to received a free pass to the event via a contact at my place of employment which was in turn actually from Hewlett Packard. Thanks, JK and HP. Much appreciated.

I arrived shortly after 8AM, registered to receive my badge and t-shirt then milled around the vendor booths until the keynotes were ready to start. I watched the keynotes (Jim Zemlin, Linux Foundation and Jim  Whitehurts, Red Hat), went to every session I could and came back to the main ballroom for the panel discussion with Jon “Maddog” Hall, Eben Moglen and Dan Frye and the following interview of Linus Torvalds by Greg Kroah-Hartman to wrap things up for day 1. So far, so good.

The Keynotes

Jim Zemlin’s opening keynote “Imagining a World Without Linux” was decent. While he did take some inevitable potshots at Microsoft, the message was generally very positive and uplifting. I won’t go into details but basically Jim described a world without Linux as one that would be black & white as opposed to the colour filled world we know today (due to Linux). Jim is a smiley and positive person on stage, his style helped kick off LinuxCon 2011 with a good vibe.

Jim Whitehurst, CEO of Red Hat had a similar approach of sending positive vibes but focused on how the progress of Linux and Open Source has enabled businesses and business models. He said that Google wouldn’t exist (at least not in it’s current form) without Linux and basically implying the same about other major well know Linux-powered companies such as Amazon and Facebook. Jim struck me as a fairly modest fellow but he wasn’t shy about mentioning Red Hat’s penetration into Fortune 500 companies. Nor was he reserved about how Linux has powered, enabled, strongly driven by or directly benefitted various global forces that may or may not be angels (U.S. Navy, NSA, Russian Military, NYSE/Wall Street). While his examples spoke to the breadth of applications for and the wide reach of Linux, I couldn’t help but think about how the pervasiveness of Linux is not only helping drive great positive change int he world but may also be powering negative forces as well.

Overall both Jim’s did a good job and left me excited for the rest of LinuxCon to come.

First Day Sessions

I attended four sessions on day 1:

  1. Centralized User Administration with FreeIPA and sssd by Stephen Gallagher
  2. Watching Mad Men and Thinking About Open Source by Karen Copenhaver
  3. 20 Years – And More – of Kernel Development by Jon Cobert
  4. What to Expect from Linux Storage by James Bottomley

Centralized User Administration with FreeIPA and sssd

My first LinuxCon session was by Stephen Gallagher of Red Hat. As is clear by the title, it was about FreeIPA and sssd, two emerging Red Hat driven projects relating to centralized directory and authentication services. Stephen wasn’t the most natural speaker I’ve had the pleasure to watch and I suspect that presentations aren’t something he does on a regular basis but he clearly knew his material and he was able to field the post-presentation questions with ease. The presentation material was fairly spot on to what I expected. I should stop by the Red Hat booth and speak with Stephen tomorrow as there are a few FreeIPA/sssd related questions I have which I didn’t ask during the question period. Overall, I was satisfied.

Watching Mad Men and Thinking About Open Source

First of all, Karen is a more natural speaker than Stephen but I suppose that’s to be expected: She is legal counsel for the Linux Foundation. The material in this session while clear and understandable was maybe not quite as impactful as I had hoped. Karen had some very nice points and brought good historical reference to the table but it wasn’t really anything that I didn’t already think think about in my own internal dialog, for the most part.

Some key points that Karen made early which did resonate with me:

  • “It’s a privilege to work on something so important”, I believe she was quoting Linus Torvalds. This hits home for me as my work is only to enable the much more important and relevant work of others.
  • The observation that the open source community generally doesn’t have time for anything but the truth which is a nice ideal but perhaps isn’t necessarily reflective of the entire open source world so much as a few of the important luminaries.
  • Identify the things that you value and… well I missed that part. But I do think identifying the things you value is, well, valuable.

These are all straight forward things but to hear someone say them can be powerful. This session was good but it wasn’t quite as hard hitting as I thought it might be based on the title and description. It was no let down, though.

20 Year – And More – Of Linux Kernel Development

Ok, now we’re getting way out of my league. Jon Corbet is a high profile Linux kernel contributor and he knows what he is talking about. This man has confidence and ostensibly the knowledge to back it up. His overview of the last 20 years of Linux kernel development was excellent and spotted with just enough humour to keep the real developers cracking up and the rest of us only getting every second joke.

Jon’s timeline approach to describing the history of kernel development was excellent and enabled him to visually map releases, events and growth in a very simple and understandable way. He made an excellent observation regarding the pace (measure by lines of code) of Linux kernel development during the dot com bust not slowing down one bit despite industry turmoil and job loss and pointed out the correlation between important points in Linux kernel development time with other events that may not be obvious to every outsider (BitKeeper, Git, time between certain releases, Merge Window, etc.).

While this session was developer focused, it wasn’t so technical to be devoid of value for anyone else, in fact I think it really helped frame the history of Linux kernel development for me in a way that I had never experienced before. Way to go, Jon.

What To Expect From Linux Storage

I’m not sure why James’ talk was titled what it was because for the best of what I could tell, the majority of the talk was about what already is, not what to expect. That’s not to say it was devoid of important information regarding “what to expect” and maybe it was because James ran out of time and had to skip some slides but I did find the title interesting in that capacity none-the-less.

James is charismatic. He makes jokes, he wears a bow tie, he speaks with an attractive accent. He’s also clearly very knowledgable about his part of the Linux kernel: the Block layer.

Being a sysadmin, knowing more about the block layer and James’ perspective on storage was hugely beneficial. He has historical reference that I never will and deep knowledge of the kernel which I’ll never achieve. With that said, some of his opinions regarding specific technologies and methods, I personally already held myself! How is it that a Linux kernel rube such as myself could had gleaned the same opinions on specific technologies as one of the people  who understands these technologies the best of anyone? iSCSI was an example. I think it’s safe to say James thinks iSCSI is an abhorrent mess that simply tries to solve a problem in entirely the wrong way. I’m also not a big fan of iSCSI and his reasoning  resonated with me, despite my lack of in depth knowledge.

I could go on because I liked this session but I already feel like I’m burning myself out on this summary of day 1 and we haven’t even gotten to the panel discussion or Linus interview yet.

Panel Discussion

The panel discussion with Jon Hall, Eben Moglen and Dan Frye was fairly profound despite Eben using the platform for an interesting but strangely placed speech that appeared entirely scripted/written. That’s not to say I didn’t like his speech or that I don’t agree with him or his world views but the way he momentarily took over the panel with what was clearly a pre-planned speech during a panel discussion main-hall format was strange indeed.

Dan Frye struck me as level-headed and one of those business people whom can take the challenge of  balancing the need to run a profitable business with social awareness and decency and excel at it. I’ve never really doubted IBM’s commitment to Linux and I know their commitment is based on profitability but the way that Dan framed the reasons that he and his team knew Linux meant good business for IBM put a smile on my face.

Jon Hall’s experience in the computing industry is staggering and humbling, even for today’s big shots. What a dude. Level head, very articulated, sense of humour and a huge white beard. It’s hard not to love the guy after watching that panel discussion. Jon talked about his hopes for how Linux and the open source model will foster the next generation of great thinkers, movers and shakers and enable them to do great things. I liked that.

I’m not really sure what to say about Eben. I agreed with everything he said but he just wasn’t as loveable as Jon Hall. Must be because he’s a lawyer :D I suppose that slightly awkward speech about the troubled times that are looming (mounting patent threats and inevitable “10-20 billion” dollar war) could have been a factor as well. That said, he seemed positive despite the heavy and serious tone he used to describe the battles ahead.

On one hand, the panel discussion left me feeling good and uplifted but on the other hand I was left with a feeling of powerlessness. I’m not one of the next great thinkers, doers or talkers. What’s my place in the Linux and open source world, then? Everything that was discussed revolved around the greatest minds in open source and the huge impacts made by major players. I almost felt a little left out as a lowly sysadmin whom has to deploy at least some non-RMS blessed systems alongside the requisite Linux systems. What’s my role in all this?

Interview with Linus

I really don’t have much to say about this one. Linus is down to earth, but strong in his opinions. He admits when something is outside of his immediate expertise, as evidenced by his answers to many non-Linux kernel specific questions. He talks well and he would have preferred if the crowd did not give him a standing ovation at the end but I suppose you cannot make a room full of Linux geeks sit down when their proverbial leader is being applauded.

I liked a lot of what Linus talked about regarding the modern direction of Linux such as the version numbering changes, the idea that we should be looking backwards at how to improve existing subsystems and layers instead of always looking forward to new feature inclusions. I liked how he described the cross-pollination of various parts of Linux that exist when everyone from embedded systems to massively parallel SMP systems are made to use the exact same kernel instead of everyone having their own specialized forks.

Linus was clam and cool, just like Linux and I had a seriously good time at LinuxCon today. Rock on, LinuxCon!

Browsing Automounted NFS with Nautilus

Has browsing automounted NFS shares with nautilus got you pulling out hair in frustration?

Ever since we transitioned from the RHEL4 environment to Fedora 14, people have been reporting terrible slowness and delays in nautilus when browsing our NFS shares. Reports of waiting over a minute for an NFS automount root-level directory with < 100 sub directories to display the contents are not good.

This wasn’t a problem on our old RHEL4 terminal server and I couldn’t for the life of me understand how nautilus could have become so slow in the years since RHEL4 was released. It just didn’t make sense. I started to think something had to be wrong and that this wasn’t just the new normal expected behaviour but I had nothing to go on.

I tried the basic recommendations: Disable thumbnails, disable preview, disable directory item counts. That didn’t help the user experience in any dramatic way. At this point, I started recommended pcmanfm and thunar as a way to workaround nautilus’ terrible performance. I even wrote a fairly concise script for modifying the default file manager and desktop-drawing application so that using a different file manager wouldn’t be so foreign in GNOME.

Then one day I started looking at the verbose level output from automount while browsing the NFS mounts with nautilus and found a substantial amount of this in the logs:

Apr 28 11:19:10 hostname automount[18959]: attempting to mount entry /home/.svn
Apr 28 11:19:10 hostname automount[18959]: key ".svn" not found in map source(s).
Apr 28 11:19:10 hostname automount[18959]: failed to mount /home/.svn

Oh my! Why are there repeated access attempts for “.svn”? What is causing automount to perform map lookups for “.svn” in the automount-controlled directories? Could it be nautilus?

Why yes!

As it turns out the GNOME SVN integration package “gnubversion” includes a nautilus extension and this extension was causing Nautilus to look for “.svn” directories everywhere and it just so happens that looking for “.svn” in a root-level automount directory causes slow map lookup failures that (presumably) kill the perceptible performance of browsing automounted NFS shares.

I removed gnubversion (as no one was using it) and the user experience for nautilus has normalized. While nautilus still isn’t as speedy as pcmanfm or thunar, its no longer a cause of forceful hair removal incidents… and all is well in the world.

Which Distro for PPC64 Server?

We (work) have two IBM p505 Express Servers.

Right now one machine is running an old way out of support RHEL4 installation and the other is on Fedora 12, which is no longer supported by the Fedora Project. Paid support/subscription is not a consideration yet for this project, but I do want to run a modern Linux distribution for the associated modern application software and maintenance.

I basically need to move these servers to something free and supportable. I’m finding out that there aren’t a lot of options in PPC Linux as when I was last interested in this architecture. It’s pretty much just:

I realize there is RHEL and SuSE Enterprise for PPC64 but those are subscription products without free binaries available. I’m not prepared to build an RPM-based distro from source at this point so I need something with binaries or something where building from source is highly automated and integrated, such as Gentoo. Digression…

The question is which of these distros do I go with? To answer the question I suppose I need to define the roles.

These two pSeries servers a redundant pair running LDAP/Auth Service, NTP, DNS and DHCP. The load is low but I want a solid modern software platform on both these servers from now until they are replaced with in the future (which is likely to be integration into a centralized architecture).

With that said, and with my familiarity level of these distros, I would first lean towards Debian and then to Gentoo and finally to CRUX PPC.

Debian is a binary distribution, which is nice for maintaining a server. Debian is more familiar to me. What are the arguments for Gentoo or CRUX PPC?

Agree or Disagree?