So what is new?

For the first time in 5.5 years, I am single and living totally, utterly alone. This is a big change that I am learning to deal with day by day.  Thankfully, I find myself busy enough that there isn’t a lot of downtime for me to dwell on negativity. I have challenged myself to use this as an opportunity to self-evaluate and change the things that I find which I do not like. I call it “Continuous Improvement”.

I recently achieved my goal of commuting by bicycle for 100 consecutive work days. I then promptly took several days off due to a series of crashes caused by the “winter” conditions here that quickly fade away. I’m now back in the saddle riding everyday and it feels great. 22km of cycling a day does the body good.

Professionally, my interest in cfengine and IT operations has been steadily increasing.

My adventures with cfengine have led to much pondering over the “meta topics” of configuration management such as the release management aspect (version controlled workflow for updating, authorizing and installing policy) and policy writing best practices such as file copy vs. file edit and policy layout/design. It is these meta topics that I find most immediately challenging when considering cfengine deployment. Learning from others, I have developed a release management workflow of sorts. Perhaps it will eventually be of quality enough to share.

With regards to IT operations, I’ve been invigorated by the LISA ’11 talks available on YouTube. These talks exposed me to several (new-ish and old) books and authors that I am now exploring and enjoying. Some favorites so far are “The Visible Ops Handbook” and “Future Shock”. I am just getting started with “Lean IT”.

Speaking of LISA, I am now registered for the Cascadia IT Conference held March 23 and 24th this year in Seattle, WA. I am super excited to meet my fellow LOPSA members (and semi-local sysadmins) that I normally only interact with via the #lopsa and #lopsa-lounge IRC channels! Hopefully the employer professional development funds can cover the registration fees… but the conference is so close to fiscal year-end and the fund might be exhausted before HR reviews my application for reimbursement. Fingers crossed.

Aside from new interests in system administration but remaining in the professional sphere, there is also an excellent potential opportunity for a new position with my current employer available to me. I don’t regard anything as set in stone so tonight I will gear up and put in my application ASAP. Some very excellent key people are waiting for it to hit their desks and I will not disappoint!

I will make no promises with regards to new content… but let it be known, techslaves.org lives!

No related posts.

– The Practice of System and Network Administration, Thomas A. Limoncelli and Christine Hogan.

Bingo. But can there be a third hybrid model?

I currently represent the decentralized model and I must agree with these two fine authors that the benefit of my close working relationship with the individual department/group is that the service provided is highly customized and focused. The central IT department(s) are understandably focused on large-scale issues (“Infrastructure”, “Communications”, “Collaboration”, “Applications”) and as such do not always represent the most ideal channel for delivery of IT services to the various research groups and departments on campus, often with more nuanced, specialized and micro-level issues.

One of my developing long-term goals is to (warning: business jargon) “bridge the gap” between the focused local support that I currently represent and the value proposition(s) of centralized IT services. I’m not yet entirely certain of how to accomplish this but I am certain that there is a way to improve the delivery of IT services to researchers across all our campuses and I want to be involved.

Does such an approach warrant the definition of a third hybrid model or is this so-called bridging of the gap already encapsulated in the model of centralized vs. decentralized?

Some of the challenges I face specifically as a “standalone” decentralized sysadmin on campus are:

• Dealing with all IT needs from desktop support to infrastructure development to data security
• Developing and maintaining vendor contacts and relationships
• No immediate peers in our environment to bounce specific ideas around with
• Weak purchasing power and negotiation leverage
• Duplication of effort
• Career progression is potentially limited
• All too easy to develop a “King of the Castle” attitude
• Complacency

Some of the concerns I hear about when introducing researchers to the idea of centralized IT support:

• General lack of trust/faith in the centralized IT department
• Perceived lack of personal attention and focus (turn around times, site knowledge, etc.)
• Perceived lack of “control” over their environment (and data!) under the centralized model
• Charge-back models for IT services are viewed as grant-unfriendly
• Physical hardware ownership appears to remain important for many researchers

Of course, this is but a snapshot of the challenges I face and the concerns I’ve been hearing but they do serve as decent examples. It must also be noted that I am seeing great progress is many of these areas already because there are very bright people here already working on these challenges. My interest in this field is absolutely not unique.

For the immediate future, I’m focusing on improving my collaborations and communication with centralized IT services by helping them out where I can and leaning on them more often for our localized problems. My hope is that by constantly forging a closer working relationship will increasingly expose me (and in turn, our group) to the benefits of the centralized IT model while providing the central IT group with greater insight into our environment and how we work.

The next steps are still a mystery to me but I’m keeping my eyes open for new opportunities to bring better IT to research.

No related posts.

Not for Domains

It’s important to keep in mind that these instructions are not for a integrating FreeIPA with a Samba domain controller but merely a Samba file server. My understanding is that FreeIPA will never conveniently/properly support the necessary bits to make it a suitable backend for a Samba 3 PDC. I believe FreeIPA will eventually look towards Samba 4 integration (using Domain trusts) for this kind of integration but don’t quote me on that. Either way, these instructions are not for Samba domain controllers, just Samba file servers.

The Assumptions

There are some basic assumptions that these instructions make.

• FreeIPA is installed and functional
• You have a general idea of how to use LDAP command line tools
• If you have a nice GUI LDAP browser, you can use it to apply the example LDIFs and edit the tree instead of the ldap CLI tools
• The LDAP commands are executed on the FreeIPA server
• Samba and FreeIPA are installed on the same server (although it shouldn’t be difficult to use TLS encryption with separate servers)
• Your LDAP suffix is “dc=domain,dc=tld”
• You know the difference between the “admin” account and the directory manager and their passwords

The Goods

Let’s not beat around the bush any further.

1. Determine your Samba server SID by executing the following command while smbd is running and jot it down:

root@ipaserver:~
# net getlocalsid
SID for IPASERVER domain  is: S-1-5-21-3180075094-3458813485-3821849995

2. With the “admin” kerberos ticket, add two attributes to “cn=ipaConfig,dc=etc,dc=domain,dc=tld” that tell FreeIPA to setup each account as a Samba account and each group as a Samba group:

ldapmodify -Y GSSAPI <<EOF
dn: cn=ipaconfig,cn=etc,dc=domain,dc=tld
changetype: modify
add: ipaUserObjectClasses
ipaUserObjectClasses: sambaSAMAccount
-
add: ipaGroupObjectClasses
ipaGroupObjectClasses: sambaGroupMapping
EOF

3. With the directory manager password and the Samba SID you jotted down from above, create an instance of the 389 DS DNA plugin that will automatically generate SIDs for your users and groups which are necessary for use with Samba:

ldapadd -x -D "cn=Directory Manager" -W <<EOF
dn: cn=SambaGroupSid,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
objectClass: top
objectClass: extensibleObject
dnatype: sambaSID
dnaprefix: S-1-5-21-3180075094-3458813485-3821849995-
dnainterval: 1
dnamagicregen: assign
dnafilter: (|(objectclass=sambasamaccount)(objectclass=sambagroupmapping))
dnascope: dc=domain,dc=tld
cn: SambaSid
dnanextvalue: 15277
EOF

The thing to note here is that the “dnaprefix” is set to the SID your jotted down… PLUS a hyphen (“-”) appended to the end!

4. Now we have to start modifying the FreeIPA API, CLI and WebUI to allow us to specify the “sambaGroupType” attribute at group creation time. We have to set “sambaGroupType” because it is a required attribute for the objectClass “sambaGroupMapping” which we are automatically adding to every group with the “ipaGroupObjectClasses” setting from earlier.

Although the value is going to be “4″ for every conceivable case in this non-domain configuration, I was not able to figure out how to make the DNA plugin insert static values like it can set incrementing values so I decided to allow setting it through the CLI and WebUS with defaults enabled instead. If anyone knows how to setup 389 to automatically add an attribute with a static value upon DN creation of DNs with specific objectClasses, please tell me.

There are a few steps required to make this CLI/UI stuff happen but the FreeIPA developers have actually made this quite simple.

The rule is: Extend the FreeIPA schema first, then the CLI, then the WebUI.

4.1. Extend the FreeIPA schema with a custom field by adding the attribute “ipaCustomFields” with a value of “Samba Group Type,sambagrouptype,true” to “cn=ipaConfig,dc=etc,dc=domain,dc=tld” with an “admin” kerberos ticket:

ldapmodify -Y GSSAPI <<EOF
dn: cn=ipaconfig,cn=etc,dc=domain,dc=tld
changetype: add
add: ipaCustomFields
ipaCustomFields: "Samba Group Type,sambagrouptype,true"
EOF

As there can only be one “ipaCustomFields” attribute, if you have multiple custom fields you need to separate each definition with a “$” like so: “Samba Group Type,sambagrouptype,true$Description,attrname,isrequiredboolean”.

4.2. Extend the CLI for groups by editing the python file “/…/site-packages/ipalib/plugins/group.py” to define the custom field and specify a default if not implicitly defined (diff):

--- group.py.orig	2011-08-15 14:59:48.570715207 -0700
+++ group.py	2011-08-16 12:43:43.493236507 -0700
@@ -118,6 +118,13 @@
             label=_('GID'),
             doc=_('GID (use this option to set it manually)'),
         ),
+        Int('sambagrouptype',
+            cli_name='sgt',
+            label=_('Samba Group Type'),
+            doc=_('Samba Group Type (default is 4)'),
+            default=4,
+            autofill=True,
+        ),
     )

 api.register(group)

Important: Restart “httpd” at this point!

4.3. Test the CLI. With an “admin” (or equivalent priv) kerberos ticket, try creating a new group:

account@ipaserver:~
$ ipa group-add testgrp --desc="Testing the group.py CLI mods"
---------------------
Added group "testgrp"
---------------------
  Group name: testgrp
  Description: Testing the group.py CLI mods
  GID: 1234500010
  Samba Group Type: 4

4.4 With the CLI functioning properly, we can move on to extending the WebUI. To extend the WebUI for group attributes, edit “/usr/share/ipa/ui/group.js” like so (diff):

--- group.js.orig	2011-08-15 10:01:28.515209121 -0700
+++ group.js	2011-08-16 13:52:59.587352034 -0700
@@ -34,6 +34,7 @@
                 column({name: 'cn'}).
                 column({name: 'gidnumber'}).
                 column({name: 'description'}).
+                column({name: 'sambagrouptype'}).
                 dialog(
                     IPA.add_dialog({
                         'name': 'add',
@@ -41,6 +42,7 @@
                     }).
                         field(IPA.text_widget({name: 'cn', undo: false})).
                         field(IPA.text_widget({name: 'description', undo: false})).
+                        field(IPA.select_widget({name: 'sambagrouptype', undo: false, options: [{label: 'Local', value: 4}, {label: 'Domain', value: 2}]})).
                         field(IPA.checkbox_widget({
                             name: 'posix',
                             label: IPA.messages.objects.group.posix,
@@ -56,6 +58,7 @@
                     }).
                         input({name: 'cn' }).
                         input({name: 'description'}).
+                        input({name: 'sambagrouptype'}).
                         input({name: 'gidnumber' }))).
         facet(
             IPA.group_member_user_facet({

And then these the WebUI to ensure that you can both see the attribute in the group list, but also add it via the select widget added to the new/edit group dialog.

That should be it. Questions, comments, suggestions, correction and more… all are welcome!

Related posts:

1. LDAP User Management Tools and User Private Groups
2. Life Support!

I arrived shortly after 8AM, registered to receive my badge and t-shirt then milled around the vendor booths until the keynotes were ready to start. I watched the keynotes (Jim Zemlin, Linux Foundation and Jim  Whitehurts, Red Hat), went to every session I could and came back to the main ballroom for the panel discussion with Jon “Maddog” Hall, Eben Moglen and Dan Frye and the following interview of Linus Torvalds by Greg Kroah-Hartman to wrap things up for day 1. So far, so good.

The Keynotes

Jim Zemlin’s opening keynote “Imagining a World Without Linux” was decent. While he did take some inevitable potshots at Microsoft, the message was generally very positive and uplifting. I won’t go into details but basically Jim described a world without Linux as one that would be black & white as opposed to the colour filled world we know today (due to Linux). Jim is a smiley and positive person on stage, his style helped kick off LinuxCon 2011 with a good vibe.

Jim Whitehurst, CEO of Red Hat had a similar approach of sending positive vibes but focused on how the progress of Linux and Open Source has enabled businesses and business models. He said that Google wouldn’t exist (at least not in it’s current form) without Linux and basically implying the same about other major well know Linux-powered companies such as Amazon and Facebook. Jim struck me as a fairly modest fellow but he wasn’t shy about mentioning Red Hat’s penetration into Fortune 500 companies. Nor was he reserved about how Linux has powered, enabled, strongly driven by or directly benefitted various global forces that may or may not be angels (U.S. Navy, NSA, Russian Military, NYSE/Wall Street). While his examples spoke to the breadth of applications for and the wide reach of Linux, I couldn’t help but think about how the pervasiveness of Linux is not only helping drive great positive change int he world but may also be powering negative forces as well.

Overall both Jim’s did a good job and left me excited for the rest of LinuxCon to come.

First Day Sessions

I attended four sessions on day 1:

1. Centralized User Administration with FreeIPA and sssd by Stephen Gallagher
2. Watching Mad Men and Thinking About Open Source by Karen Copenhaver
3. 20 Years – And More – of Kernel Development by Jon Cobert
4. What to Expect from Linux Storage by James Bottomley

Centralized User Administration with FreeIPA and sssd

My first LinuxCon session was by Stephen Gallagher of Red Hat. As is clear by the title, it was about FreeIPA and sssd, two emerging Red Hat driven projects relating to centralized directory and authentication services. Stephen wasn’t the most natural speaker I’ve had the pleasure to watch and I suspect that presentations aren’t something he does on a regular basis but he clearly knew his material and he was able to field the post-presentation questions with ease. The presentation material was fairly spot on to what I expected. I should stop by the Red Hat booth and speak with Stephen tomorrow as there are a few FreeIPA/sssd related questions I have which I didn’t ask during the question period. Overall, I was satisfied.

Watching Mad Men and Thinking About Open Source

First of all, Karen is a more natural speaker than Stephen but I suppose that’s to be expected: She is legal counsel for the Linux Foundation. The material in this session while clear and understandable was maybe not quite as impactful as I had hoped. Karen had some very nice points and brought good historical reference to the table but it wasn’t really anything that I didn’t already think think about in my own internal dialog, for the most part.

Some key points that Karen made early which did resonate with me:

• “It’s a privilege to work on something so important”, I believe she was quoting Linus Torvalds. This hits home for me as my work is only to enable the much more important and relevant work of others.
• The observation that the open source community generally doesn’t have time for anything but the truth which is a nice ideal but perhaps isn’t necessarily reflective of the entire open source world so much as a few of the important luminaries.
• Identify the things that you value and… well I missed that part. But I do think identifying the things you value is, well, valuable.

These are all straight forward things but to hear someone say them can be powerful. This session was good but it wasn’t quite as hard hitting as I thought it might be based on the title and description. It was no let down, though.

20 Year – And More – Of Linux Kernel Development

Ok, now we’re getting way out of my league. Jon Corbet is a high profile Linux kernel contributor and he knows what he is talking about. This man has confidence and ostensibly the knowledge to back it up. His overview of the last 20 years of Linux kernel development was excellent and spotted with just enough humour to keep the real developers cracking up and the rest of us only getting every second joke.

Jon’s timeline approach to describing the history of kernel development was excellent and enabled him to visually map releases, events and growth in a very simple and understandable way. He made an excellent observation regarding the pace (measure by lines of code) of Linux kernel development during the dot com bust not slowing down one bit despite industry turmoil and job loss and pointed out the correlation between important points in Linux kernel development time with other events that may not be obvious to every outsider (BitKeeper, Git, time between certain releases, Merge Window, etc.).

While this session was developer focused, it wasn’t so technical to be devoid of value for anyone else, in fact I think it really helped frame the history of Linux kernel development for me in a way that I had never experienced before. Way to go, Jon.

What To Expect From Linux Storage

I’m not sure why James’ talk was titled what it was because for the best of what I could tell, the majority of the talk was about what already is, not what to expect. That’s not to say it was devoid of important information regarding “what to expect” and maybe it was because James ran out of time and had to skip some slides but I did find the title interesting in that capacity none-the-less.

James is charismatic. He makes jokes, he wears a bow tie, he speaks with an attractive accent. He’s also clearly very knowledgable about his part of the Linux kernel: the Block layer.

Being a sysadmin, knowing more about the block layer and James’ perspective on storage was hugely beneficial. He has historical reference that I never will and deep knowledge of the kernel which I’ll never achieve. With that said, some of his opinions regarding specific technologies and methods, I personally already held myself! How is it that a Linux kernel rube such as myself could had gleaned the same opinions on specific technologies as one of the people  who understands these technologies the best of anyone? iSCSI was an example. I think it’s safe to say James thinks iSCSI is an abhorrent mess that simply tries to solve a problem in entirely the wrong way. I’m also not a big fan of iSCSI and his reasoning  resonated with me, despite my lack of in depth knowledge.

I could go on because I liked this session but I already feel like I’m burning myself out on this summary of day 1 and we haven’t even gotten to the panel discussion or Linus interview yet.

Panel Discussion

The panel discussion with Jon Hall, Eben Moglen and Dan Frye was fairly profound despite Eben using the platform for an interesting but strangely placed speech that appeared entirely scripted/written. That’s not to say I didn’t like his speech or that I don’t agree with him or his world views but the way he momentarily took over the panel with what was clearly a pre-planned speech during a panel discussion main-hall format was strange indeed.

Dan Frye struck me as level-headed and one of those business people whom can take the challenge of  balancing the need to run a profitable business with social awareness and decency and excel at it. I’ve never really doubted IBM’s commitment to Linux and I know their commitment is based on profitability but the way that Dan framed the reasons that he and his team knew Linux meant good business for IBM put a smile on my face.

Jon Hall’s experience in the computing industry is staggering and humbling, even for today’s big shots. What a dude. Level head, very articulated, sense of humour and a huge white beard. It’s hard not to love the guy after watching that panel discussion. Jon talked about his hopes for how Linux and the open source model will foster the next generation of great thinkers, movers and shakers and enable them to do great things. I liked that.

I’m not really sure what to say about Eben. I agreed with everything he said but he just wasn’t as loveable as Jon Hall. Must be because he’s a lawyer :D I suppose that slightly awkward speech about the troubled times that are looming (mounting patent threats and inevitable “10-20 billion” dollar war) could have been a factor as well. That said, he seemed positive despite the heavy and serious tone he used to describe the battles ahead.

On one hand, the panel discussion left me feeling good and uplifted but on the other hand I was left with a feeling of powerlessness. I’m not one of the next great thinkers, doers or talkers. What’s my place in the Linux and open source world, then? Everything that was discussed revolved around the greatest minds in open source and the huge impacts made by major players. I almost felt a little left out as a lowly sysadmin whom has to deploy at least some non-RMS blessed systems alongside the requisite Linux systems. What’s my role in all this?

Interview with Linus

I really don’t have much to say about this one. Linus is down to earth, but strong in his opinions. He admits when something is outside of his immediate expertise, as evidenced by his answers to many non-Linux kernel specific questions. He talks well and he would have preferred if the crowd did not give him a standing ovation at the end but I suppose you cannot make a room full of Linux geeks sit down when their proverbial leader is being applauded.

I liked a lot of what Linus talked about regarding the modern direction of Linux such as the version numbering changes, the idea that we should be looking backwards at how to improve existing subsystems and layers instead of always looking forward to new feature inclusions. I liked how he described the cross-pollination of various parts of Linux that exist when everyone from embedded systems to massively parallel SMP systems are made to use the exact same kernel instead of everyone having their own specialized forks.

Linus was clam and cool, just like Linux and I had a seriously good time at LinuxCon today. Rock on, LinuxCon!

Related posts:

1. Is Ubuntu Ready for the Enterprise?
2. LVM filters and initrd

From http://support.microsoft.com/kb/314470:

System Volume
The system volume refers to the disk volume that contains the hardware-specific files that are needed to start Windows, such as Ntldr, Boot.ini, and Ntdetect.com.

On computers that are running the Intel x86 line of CPU processors and later versions, the system volume must be a primary volume that is marked as active. This requirement can be fulfilled on any drive on the computer that the system BIOS searches when the operating system starts.

The system volume can be the same volume as the boot volume. However, this configuration is not required.

and

Boot volume
The boot volume refers to the disk volume that contains the Windows operating system files and the supporting files. By default, the Windows operating system files are in the WINDOWS folder, and the supporting files are in the WINDOWS\System32 folder.

The boot volume can be the same volume as the system volume. However, this configuration is not required.

There is only one system volume. However, there is one boot volume for each operating system in a multiboot system.

So… the “system volume” is the volume that contains the boot files and the “boot volume” is the volume that contains the system files. It might have been opposite day when this was named. Yikes.

Thanks for the laugh, Microsoft.

Related posts:

1. Microsoft Campus Agreement Double Dipping
2. Fresh Win2k Install and Windows Update Error
3. LVM filters and initrd

Ever since we transitioned from the RHEL4 environment to Fedora 14, people have been reporting terrible slowness and delays in nautilus when browsing our NFS shares. Reports of waiting over a minute for an NFS automount root-level directory with < 100 sub directories to display the contents are not good.

This wasn’t a problem on our old RHEL4 terminal server and I couldn’t for the life of me understand how nautilus could have become so slow in the years since RHEL4 was released. It just didn’t make sense. I started to think something had to be wrong and that this wasn’t just the new normal expected behaviour but I had nothing to go on.

I tried the basic recommendations: Disable thumbnails, disable preview, disable directory item counts. That didn’t help the user experience in any dramatic way. At this point, I started recommended pcmanfm and thunar as a way to workaround nautilus’ terrible performance. I even wrote a fairly concise script for modifying the default file manager and desktop-drawing application so that using a different file manager wouldn’t be so foreign in GNOME.

Then one day I started looking at the verbose level output from automount while browsing the NFS mounts with nautilus and found a substantial amount of this in the logs:

Apr 28 11:19:10 hostname automount[18959]: attempting to mount entry /home/.svn
Apr 28 11:19:10 hostname automount[18959]: key ".svn" not found in map source(s).
Apr 28 11:19:10 hostname automount[18959]: failed to mount /home/.svn

Oh my! Why are there repeated access attempts for “.svn”? What is causing automount to perform map lookups for “.svn” in the automount-controlled directories? Could it be nautilus?

Why yes!

As it turns out the GNOME SVN integration package “gnubversion” includes a nautilus extension and this extension was causing Nautilus to look for “.svn” directories everywhere and it just so happens that looking for “.svn” in a root-level automount directory causes slow map lookup failures that (presumably) kill the perceptible performance of browsing automounted NFS shares.

I removed gnubversion (as no one was using it) and the user experience for nautilus has normalized. While nautilus still isn’t as speedy as pcmanfm or thunar, its no longer a cause of forceful hair removal incidents… and all is well in the world.

Related posts:

1. RHEL/CentOS, NFS and Firewalls
2. Is Ubuntu Ready for the Enterprise?
3. POSIX Default ACLs, umask and Project Directories

So why bother with knowing anything about this whole plotting thing? It’s clear how it can be useful in monitoring-style applications where data points are collected over time and then visualized via a plot or graph. Such plotting exposes trends in our environments and that’s usually a helpful tool to have around. Of course, there are other more specific problems and/or questions where collecting, plotting and analyzing data is very helpful as well. I will do my best to describe one such example.

Over the last few days I’ve been trying to find an answer to the question:

“Does the VPN add latency to our remote NX connections and if so, is it significant?”

This is a question where I believe plotting data will prove useful. There are some other sub-questions I’d like answered as well but that is the overarching issue at hand. I realized that this would be a great opportunity to re-learn some of the basics and maybe try out a few new tools at my disposal so I decided to document my journey through this foreign land for all to criticize and enjoy.

! Scientific Method

Of course, I’m not following a strict scientific method with this endeavor. The question simply doesn’t warrant an entire drawn out, highly statistically relevant result despite my best intentions in delivering exactly that. What I’m trying to do is get an accurate sense more than an exact measurement, as flawed as that might be. It’s all I can justify in terms of time and effort for this project. From that strictly academic point of view, I’m sure to fail. My hope is that the results will be pseudo-science’d enough to provide confidence in my answer and that I’ll improve my skills throughout the exercise.

What Tests?

In order to determine if the VPN is affecting our latency I need at least two tests:

1. NX connection without VPN
2. NX connection with VPN

But while I’m at it, I figured I would gather additional data in order to attempt an answer at other RTT related questions. Adding additional tests based on client system “location” (local LAN, local wireless, various locations on campus wireless, home internet connection, etc. and NX compression settings (MODEM, ISDN, ADSL, WAN and LAN) greatly increases the amount of testing required but will provide for richer data to visualize.

On top of that , each one of these additional variables I am testing is to also be tested with and without VPN. To add even more tests, each one of these combination of tests needs to be performed multiple times in order to normalize the data and to increase the statistical relevance. More samples = better data = more accurate results (at least this is the hope).

Data Collection

In order to start analyzing data, I need data. And that data needs to of be quality. And to have quality of data, I need multiple samples. And to make useful comparisons I need multiple variable data sets and at least one control data set. For all that to work, I needed a reproducible set of actions to generate traffic, collect data and extract the relevant parts.

My basic method is as follows:

1. Configure wireshark or tcpdump on the remote host to capture packets related to the NX/SSH connection that we are testing. Capture filters are used to prevent capture of any other packets.
2. Initiate NX connection to remote host (login)
3. Perform predefined action X on remote host via NX
4. Logout of NX connection from remote host
5. Stop and save packet capture
6. Export RTT statistics from capture file with tcptrace
7. Extract only the RTT data from `tcptrace` output (discard the TCP sequence # column because the absolute value doesn’t matter, we’ll use the index for the x-axis)
8. Label and save extracted RTT data as txt format for input to plotting function

Plot Types

There are two primary plot types that are going to help me answer the question at hand: scatter plots and histograms.

Scatter plots are basically used to visualize at least one data set with two display values. In this case, plotting the round trip time (RTT) in milliseconds by the corresponding TCP sequence number for various data sets. What’s more interesting though is juxtaposing combinations of data sets against each other in order to quickly visualize and observe qualitative differences.

Histograms are a way of visualizing the distribution of data set. In this case, a histogram will plot the number of TCP sequences at each millisecond increment in the data set. Visualizing the distribution of our data set will help to clarify what the least to most frequent round trip times are, something which cannot be quickly visualized in a dense scatter plot.

Looking Forward to Part 2

Now that you’ve made it through the snooze-fest that was part 1, I hope you’re eager for part 2! Oh boy! More blabbering, right? Hopefully not. Part 2 is where I’ll share some scripts, tips, techniques and finally, some finished plots for all to behold. You know, the technical stuff that we all love.

It shall be grand, now I just need to write it…

Comments are highly welcome.

Related posts:

1. Migration Weekend

Right now one machine is running an old way out of support RHEL4 installation and the other is on Fedora 12, which is no longer supported by the Fedora Project. Paid support/subscription is not a consideration yet for this project, but I do want to run a modern Linux distribution for the associated modern application software and maintenance.

I basically need to move these servers to something free and supportable. I’m finding out that there aren’t a lot of options in PPC Linux as when I was last interested in this architecture. It’s pretty much just:

• Debian
• CRUX PPC
• Gentoo

I realize there is RHEL and SuSE Enterprise for PPC64 but those are subscription products without free binaries available. I’m not prepared to build an RPM-based distro from source at this point so I need something with binaries or something where building from source is highly automated and integrated, such as Gentoo. Digression…

The question is which of these distros do I go with? To answer the question I suppose I need to define the roles.

These two pSeries servers a redundant pair running LDAP/Auth Service, NTP, DNS and DHCP. The load is low but I want a solid modern software platform on both these servers from now until they are replaced with in the future (which is likely to be integration into a centralized architecture).

With that said, and with my familiarity level of these distros, I would first lean towards Debian and then to Gentoo and finally to CRUX PPC.

Debian is a binary distribution, which is nice for maintaining a server. Debian is more familiar to me. What are the arguments for Gentoo or CRUX PPC?

Agree or Disagree?

No related posts.

Three major things happened that made it possible to compromise:

1. I received an OK from the higher-ups that indeed it would be ok to mandate that users who want remote access to our system have all their Internet traffic routed via the VPN.

2. The VPN configuration that was proposed is not that of a private group but the generic campus-wide VPN solution with ACLs to allow VPN clients to access our resources.

3. The generic campus-wide VPN service does *not* implement any outgoing restrictions, unlike the private VPN groups do. This makes it possible for VPN users to do whatever they want when connected, with the stipulation that it’s akin to bringing your personal computer to campus in terms of privacy.

This really is the best of both worlds (security, ease of use). The campus VPN service is super-simple to use and we restrict access to our services to VPN users instead of the whole Internet. While we don’t control who gets access to the VPN, the pool of users is MUCH smaller and at least semi-trusted by the organization. Thus the risk is greatly reduced.

I’m happy.

Related posts:

1. Remote Access Solution

I’m in a bit of a pickle.

Traditionally, we’ve always allowed wide-open SSH access from anywhere to our main terminal server for remote access. Since we use NX (neatx, FreeNX, NXclient, etc.), all we ever needed open was SSH to make it all work nicely. Sure, SSH is a big bruteforce target but with DenyHosts and low thresholds things are pretty well under control. I realize huge distributed bruteforce attacks are still possible against a DenyHosts protected SSH daemon but we have to factor in ease of use when thinking about security and the low risk of massively distributed bruteforce attacks.

With the deployment of a new terminal server we have the opportunity to use the Cisco-based campus VPN service delegated to us via a customized VPN group. This is all well and good except for one thing: There is no ability to set custom routes for the clients based on those VPN groups. When a remote user connects to the VPN, *all* their network traffic has to be routed via the VPN. This typically makes sense, it is a VPN after all. However, we have a strong use case for which this simply won’t work. We have remote collaborators from anywhere in the world and mandating that they route all their Internet traffic via us when they need to remote in to our systems is unacceptable. Not to mention we have local users working from home and the same applies. We cannot mandate that these users have to route all their non-work related private traffic via us whenever they also need to access our resources at the same time.

Alternatives

So what are we left with? What can we do to provide increased security while also maintaining the ease of use of direct SSH access? I’ve thought of a couple things, but they also have disadvantages:

1. Run our own VPN with OpenVPN on dd-wrt or a generic box.

This sounds great at first but this requires that we support another box and another service directly. The campus VPN service is managed by the central IT group, our own VPN would add support costs of our own and specifically with the client installation side of things. I’d have to come up with our own instructions for installing and configuring the clients on multiple operating systems and hope that our users don’t have serious problems getting it working.

2. Use SSH but mandate the use of SSH keys.

No doubt some of our users have bad passwords and using SSH keys would prevent password bruteforce attempts including massively distributed ones. But mandating the use of SSH keys seems like hell from a support perspective. Asking users to generate keys and have them available on any client they wish to use is really pushing the boundary of what they’ll be able to successfully do on their own. I just know I’d get the evil eye from everyone if I handed them the instructions for how to accomplish this.

Push Back Times Two

What makes this worse is no matter what I choose, I’ll get push back from one side or the other. Make things more secure but more complicated and my user base will be seriously unhappy. Continue allowing direct SSH access as the long term policy and the central IT group(s) will tar and feather me: “Bad practice! Bad practice!”. Damn this pickle!

So, again, what are we left with for secure remote access solutions that are both secure and simple enough for anyone to use? Any suggestions dear Internet?

Related posts:

1. Remote Access Solution: Follow Up