Tag: SSH

Tempin' Ain't Easy

I try not to think about the fact that it's been seven years since I got my CS degree and I haven't put it to use professionally.

I entered the field at the wrong time and in the wrong place. It's rough all over, and the housing bubble hit Arizona disproportionately hard. I've spent the past few years working as a temp and building the odd website on the side.

The first temp gig lasted two years -- ironically, longer than any other job I've had. But I got laid off about a year ago.

There's this kind of paranoia you get. It could happen again any time. And it has absolutely nothing to do with how hard you work or how good a job you do. You could be out on your ass tomorrow, on the whim of some guy you've never met.

I've heard some of the "get a job" rhetoric lately and it's just baffling. A hell of a lot of people would like very much to get a job. I've been either unemployed or underemployed my whole adult life, and that's with a degree that, fifteen years ago, could have gotten me six figures.

Not that I intend this as a pity party. I've got work now, and it pays well enough to live comfortably while still squirreling away enough each week that I'll be okay for a few months if I find myself unemployed again. There are a lot of guys who have it a lot worse than I do.

And if you take anything away from this comedy of errors, let it be that: this is the story of a guy who's doing okay in this economy.

Job 1: Fortune 500 Company, Real Estate Business

Job: Imaging laptops, working in a warehouse, inventory duty
Distance from Home: 3.5 miles
Best Thing: Laid-back atmosphere most of the time
Worst Thing: Lung fungus
Length of Service: 2 years

This wasn't a bad gig, really. Not intellectually challenging, but I worked with some good people, I got some good exercise in, and most times things were pretty laid-back.

But it wasn't worth giving up my health for, and ultimately that's what I did.

I did a lot of work out in a dusty warehouse, and I managed to contract valley fever. For those of you not from around here, valley fever is a lung fungus, and it lives in dust. The Valley and valley fever are like the Internet and Hitler comparisons -- you stay there long enough, it's something you're eventually going to have to deal with.

So I contracted a lung fungus working there, and I've still got asthma. It's manageable now, but I'm not what I was. Before I took that job I was healthy.

The next-worst thing about the job, after the lung fungus, was the meddling from up the chain. People with little-to-no grasp of our actual day-to-day operations had very strong opinions of what those operations should be, and precisely which boxes we should check on which forms each and every single time we did them. Precisely what those opinions were tended to change from week-to-week, producing an ever-changing, increasingly complex system for dealing with very simple tasks.

And as this went on, the environment became less and less laid-back, and more and more stressful.

There was a real disconnect between the building I was in and management out on the west coast. Within my office I was regarded as an essential member of the team, and indeed my bosses not only recognized my value, they realized that I could probably be doing more for the company than just counting how many sticks of RAM were left in inventory, and fought hard to get me not only hired on but promoted.

It's no small comfort to me that every single person who actually worked with me was pulling for me. To the point that when Corporate decreed that all the temps would be let go, my boss's boss's boss got reassigned for telling his boss's boss's boss exactly how he felt about that.

It was nothing personal. And it was nothing to do with my performance. I was just caught up in a bloodbath. I was part of the first wave, but it kept going. Last I heard, they'd laid off another third of my department, every help desk tech in Arizona, nearly everyone in the front office, and most of the people up the chain to VP. And demoted my boss back down to tech.

But before all that, I got a layoff for Christmas. I lost my job two years, to the week, after I'd gotten it.

There's a fatalism that kicks in after awhile. A knowledge that no matter how hard you work and how much you're appreciated, there's some clown in a corner office somewhere who's never met you but has the power to decide whether you're drawing a paycheck next week.

But ultimately there's something liberating about that, too. After awhile you stop trying to impress the clowns in the corner offices who have never met you. You realize the only people worth giving two shits about are the ones you deal with every day -- and that trying to impress them isn't about whether you'll have a job next week, it's about doing a good job for its own sake and for the sake of your team.

Those guys had my back. And that means more to me than a paycheck ever did.

Unemployment

Unemployment sucks. But it could be worse.

It's a pretty damn smooth process in this day and age -- all online, no driving across town and waiting in line. You fill out an online form, they take a week or two to make sure your story checks out, and then they open up a bank account for you, send you a card, and put money in every week.

Once a week you'll have to resubmit your claim. You tell them you're still looking for work (and keep evidence on file in case they ask for it -- I kept rather a long Excel spreadsheet with a list of everybody I'd contacted) and declare any money you've earned.

The whole thing's demoralizing and more than a little Kafkaesque -- Ursula K Le Guin recently described it quite wonderfully in a short story called Ninety-Nine Weeks: A Fairy Tale, and it's barely an exaggeration. That spreadsheet I mentioned where I kept track of all the dozens jobs I applied for? Only one of them ever actually got me an offer, and it was out-of-state -- more on that below. By the time I did finally get work again, it wasn't from the job search, it was from the same temp agency I'd been working for since '08.

Job 2: Local Non-Profit, Medical Industry

Job: Imaging laptops
Distance from Home: 13 miles
Best Thing: A job!
Worst Thing: Poor pay, sporadic availability
Length of Service: 3 months, off and on

This one wasn't too bad either. Neat office, nice people, and a certain degree of autonomy. The cramped little room I worked in got pretty crowded and hot as time went on, and there was a whole lot of downtime as I waited for laptops to finish imaging, but hey, I got time to catch up on my reading.

I also learned some interesting things about security policy. I've never had to lock things down so tightly from the BIOS -- a unique strong boot password on every machine, USB boot disabled, Bluetooth disabled, and on and on.

The toughest thing was that this wasn't a 40-hour-a-week job. It was "We just got these laptops in; image them and when you're done we'll send you home and call you back in when we get more."

And, without getting into the specifics of my pay, here's where that got frustrating: often I didn't make significantly more money than if I'd just stayed at home and collected unemployment.

Unemployment in Arizona works like this: you get a weekly stipend of up to $240. I was eligible for that maximum amount.

Every week, you report how much you've earned. You can earn up to $30 before they start subtracting your earnings from your unemployment check.

So there's this sort of dead zone between $30 and $270 where you are making the same amount of money whether you work or not.

And at this job, I frequently worked a weird part-time schedule and fell into that zone. Once I got past that first $30, I wasn't actually making any money; I was just getting a paycheck from the temp agency instead of the state.

Obviously there are still reasons to work. For its own sake, first of all. And second, to stay eligible for my healthcare, which was set to expire after three months without work. (I got back into the market just in time, but not fast enough to keep someone from fucking up my paperwork and taking me off their books even though I was still paying in every week. I had to call three different departments to get it corrected and my last prescription covered.) But there's still a definite sense of frustration in knowing that you're effectively working for free.

More than one other tech actually told me I should slow down and deliberately take longer to do the work so that I wouldn't get sent home in the middle of the week to await the next shipment. What a position to be in -- effectively being punished for being efficient, and incentivized to slow down and waste time.

This, as you will see, was to become a recurring theme.

Job 3: Company You've Probably Heard Of If You Live in North America, Retail Business

Job: Phone support
Distance from Current Home: 30 miles
Distance from Apartment Where I Lived 4 Years Ago: Directly across the street
Best Thing: Coworkers seem like all right guys
Worst Thing: The single worst job I have ever had. Fuck these people.
Length of Service: About a month

On some level, this fucking fiasco was my own doing.

I'd been poking through listings on some job site or other (probably not CareerBuilder; I quit using it after I discovered it was the thing that kept locking up my browser and hanging my entire system) and I noticed an IT job being offered through my temp agency which my rep hadn't brought to my attention. So I E-Mailed him and asked about it. In hindsight, I should have assumed there was a good reason he hadn't approached me about it.

It was phone support. Not phone support like I'd done before, but in a phone bank -- I had a few feet of shelf that I wouldn't really refer to as a desk, partitioned off from the guys next to me by small dividers that I wouldn't really refer to as a cubicle. Every morning at 6 AM I pulled up whatever broken chair nobody was sitting in, put on a headset if it was still where I'd left it the day before, and started working my way through a list of branches to call to walk their managers through installing new kiosks that didn't work very well in buildings that, half the time, weren't cabled correctly. (Ever walk a retail manager through recabling a patch panel? I've done it six times before breakfast.) It was dimly lit and it was dehumanizing -- I'd compare it to an assembly line, but the assembly lines I've seen are a whole lot livelier and more fun.

(I will grant one thing to the "cog in a corporate machine" setup: this is a company with hundreds of stores, all organized exactly the same. Each store has the same patch panel with the same numbered ports that go to the same rooms and assign IP's based on the same scheme. There was this in-house .NET program we had that would let you plug in a store number, automatically populate the IP address for every port in the place, and give you a one-click ping for each one. That's the advantage of a company that treats its stores as unifom, cookie-cutter widgets. The disadvantage is that it treats people exactly the same way.)

I spent most of each day on hold listening to the same fucking 16 bars of piano music over and over again. Periodically interrupted by a recorded voice telling me I was on hold, of course -- and if I ever meet the son of a bitch who decided to stick voice recordings in the middle of hold music, I am going to gouge his eyes out with my thumbs. I know I'm on hold, asshole; that's why there is music playing. About the only thing that could trick me into thinking that I wasn't on hold would be if the music abruptly stopped and I heard a human voice instead.

There were a couple of guys there who I'd gone to high school with. One of them I recognized but hadn't really known very well; the other used to pick on me but claimed not to remember me (he blamed it on the drugs he'd been doing back then and I am inclined to believe him). Now, remember how earlier I expressed frustration that my career hasn't really gone anywhere? Well, if you want a symbol that will hammer that little insecurity home, suddenly finding yourself sitting next to a couple of guys from high school is a pretty good one. But probably not as good as being directly across the street from the apartment where you lived back when you worked a previous dead-end job. Man, that would have been a sweet commute in 2007!

So no, let's say that this job wasn't the best fit for me. But dammit, I got up every morning at 4:30, put on a smile, went in, did my job and did it well. I blew through every task they gave me and asked for more.

This, as it turned out, was a problem. But nobody ever actually bothered to tell me that.

One morning I walked in and found that my login wasn't working. I asked the guy who'd been training me; he hemmed and hawed and wandered off for awhile, then came back and told me to turn in my badge.

It bears repeating, at this point, that I had just driven 30 miles to show up to work at 6 AM.

My rep told me that they'd called his office the previous evening to tell him to call me and tell me not to come in to work in the morning -- after he'd already gone home for the day.

He added that I'd been sacked because they thought I didn't schmooze enough with the end users over the phone -- something that nobody had ever actually complained to me about. I wasn't rude, or even brusque; I was just, in my rep's words, "too focused on getting the job done". I'm used to support jobs emphasizing getting the task done quickly, because the user doesn't want to be on the phone and wants to get back to what she was doing. But apparently that's not how it worked at this company; they wanted me to slow down and shoot the breeze -- except nobody ever bothered to tell me that. Come on, guys, if you want me to talk about the weather, just say so -- I have quite a lot to say about the weather in Phoenix in June, even when half the state isn't on fire.

Anyhow, it's the only job I've ever been fired from. And nobody even bothered to tell me there was a problem, let alone that I'd been fired.

The guy who walked me to the door was apologetic and told me not to worry about it, that people get fired from that place all the time through no fault of their own; maybe just for looking at somebody the wrong way. And it occurred to me that I'd passed my boss early one morning in the hall and, when she asked how I was doing, cracked a grin and responded "Hanging in there" -- and she apparently took offense that I hadn't said something more enthusiastic.

On the whole, pretty demoralizing and upsetting, and far and away the worst professional experience I have ever had.

Of course, I use the term "professional" in its loosest possible sense.

Job Interviews

Through it all, of course, I was interviewing wherever I could.

There are lots of stories I could tell. The temp agency I spent half an hour trying to find. The interview where I referred to a former coworker as "A temp like me, but kind of a slacker" but the interviewer just caught the "like me, kind of a slacker" part and that pretty well torpedoed me. The interviewer who asked me about a comment I'd posted about Spore's DRM on the FTC website back in '09 and then followed up by asking my opinion about SB1070. But the best story is the hosting company I saw advertised on a billboard.

"Do you know Linux? We're hiring!" said the billboard, with a colorful mascot next to the words. I would see it on the freeway on my way to work. Or maybe it was on my way home from work. Maybe it was both; I think they had more than one billboard.

Well, hell yeah I know Linux. I pulled up the website and submitted a resume. Turned out it was a hosting company -- even better. I spent most of '07 running the backend of a local ISP singlehandedly; I know my way around Apache httpd and MS IIS pretty well.

So they called me back, and the most immediately odd thing was that they told me the job was in Austin. Why would a company in Austin advertise in Phoenix?

Well, of course the answer is that they couldn't find anybody in Austin willing to accept the shitty salary they want to pay for Linux administration, so they're advertising in depressed markets that are full of desperate, unemployed Linux admins. But as you might expect, they didn't come right out and say that.

No, they gave me some talk about how they're expanding into new markets, and how they'd pay for my relocation, and they didn't balk when I gave them a deliberately high figure for my expected salary. They made the whole process seem very exclusive, putting me through three different interviews -- a general one, a second one with a series of technical questions, and a third where they had me SSH into one of their servers and demonstrate that I know my way around bash.

And then they offered me an hourly rate that was maybe fifty cents better than what I was currently getting in the phone bank. And a relocation fee that might have covered a U-Haul rental, deposit, and first month's rent on an apartment.

I hear Austin is a neat place, but no thank you.

It was about this point that I decided to read some employee testimonials on the place, and it sounded suspiciously like the terrible job I was already working at.

The billboards are down now. I wonder if they ever found anybody desperate or gullible enough to take their offer.

Job 4: Contractor for a Contractor for a Contractor, Insurance Industry

Job: Imaging laptops
Distance from Home: 32 miles
Best Thing: Getting work immediately after the previous fiasco; autonomy and people who were happy to see me
Worst Thing: Night crew fired after their first day
Length of Service: 6 weeks

Actually, before this job my rep sprang into action and got me a half-day gig fixing a company's QuickBooks setup, a mere 5 days after the debacle at my previous job. But I'm not counting that as its own section. My rep's cool, though.

Anyhow, shortly after the half-day QB fixer-upper, he found me something else and, at last, I got to be part of a Windows 7 refresh -- the precise thing that my boss, the previous December, had assured me would ensure my job security for another year, the week before announcing that the Windows 7 rollout had been canceled and so had my employment.

Anyhow, this one was interesting. The idea was to provide a minimum of disruption for the employees, while upgrading most of the office to Win7 in a matter of weeks.

So we had a night crew. They came in, ran a script to back up the user's files, either reimaged the user's existing computer or grabbed a new, freshly-imaged one that I'd already put together, restored from backup, and left it to me to walk the user through initial configuration the next morning.

At least, that's how we eventually got it working. The first night, things failed rather spectacularly.

I got in the next morning to find the night crew still there, a small handful of computers actually in working condition, and the rest in various states of completion.

The way I heard the story went something like this: one tech on the crew had asked the guy in charge what the plan was -- how they were going to split up the workload, what the schedule was, etc. He had made some vague "Just get started" noises. She asked him a few more times; he responded similarly. Finally she just went to work; she was responsible for the handful of machines that had actually been finished, while the other techs hadn't really worked out a plan for how to get their work done.

So the company fired everyone else and put her in charge of the new team.

After that it went really smoothly most nights. There were a couple exceptions -- one weekend when the generator had to be turned off for maintenance and so they couldn't come in to get computers ready for Monday, and one night when the AC was out and it was too hot to work. But no more problems from the techs themselves; the second crew did a really great job and made my life much easier.

Job 5: Company You've Probably Heard Of If You Live in the Southwestern US, Real Estate Business

Job: Imaging laptops
Distance from Home: 22.5 miles
Best Thing: Autonomy
Worst Thing: Still a bit of a drive.
Length of Service: 4 months so far, out of a one-year contract.

And from there I moved on to my fifth job of the year, not including freelance Web design or that one-day gig fixing QuickBooks.

This one comes with a one-year contract, so hopefully that'll hold and I'll still be there through next August. But I'm not going to take that for granted; one of the many lessons I learned in the Dank Pit of Phone Support last summer is that a six-month contract can turn into a one-month contract with absolutely no warning. Course, I've been working this one long enough that I am confident in saying that this time I am working for decent human beings, but again, it's not the people I've actually met I'm worried about. And every time I hear the Windows 7 rollout's been delayed, I get a little nervous.


I guess it's worth asking, what motivates me to come to work every day and do a good job? Here's what I can come up with:

  • Need for money
  • Need for health insurance
  • Pride
  • Loyalty to my coworkers

It's instructive to note the things that aren't on the list. "Hope for promotion" and "fear of losing my job" are conspicuously absent -- yes, I do feel both of those things, but as I've mentioned several times, I have absolutely no sense that my employment or advancement is tied to my performance in any way. They're motivating factors just as much as the potential for finding a $100 bill on the ground or tripping and cracking my skull -- they're both things that have some potential for happening, and my job performance has about as much to do with the likelihood of either one.

Also missing: "company loyalty". And unlike those other two things, this isn't something I have in the slightest. I am, as I said, loyal to my coworkers, and I appreciate my rep at the temp agency, but that's not the same thing as being loyal to either the company I'm working for or the company that placed me there. If I get a better offer I'll take it -- and those last two bullet points are the only reasons I'll give two weeks' notice.

On the whole I'm not entirely sure this is a bad thing from my perspective -- hell, the ideal list would probably have two bullet points instead of four. Company loyalty, the stick of firing and the carrot of advancement -- I don't need those things to do a good job. But from the company's perspective, it's probably a bad thing.

And if I may be so bold, I think I'm probably representative of a good solid chunk of my generation. Educated, underemployed, unable to hold down a job for more than two years through no fault of my own -- what happens when that's your workforce? In the coming decades we're going to find out.

Unison: File sync from Ubuntu to Windows 7

Hey, been awhile. Have been ignoring the blog (even my traditional New Year's Eve Post) and many of my other Internet habits in favor of various projects I've been hard at work on. I just pulled off a WordPress update; you're reading this so it looks like it went smoothly.

Anyhow. One of the aforementioned projects (and the thing you came here to read, if you found this page by Googling an error message -- and if you did, you may want to skip my meandering explanation and go straight for the numbered steps at the bottom of this post): I recently decided to set up a file sync system across the computers in my house. It's useful for syncing things like savegames, RSS feeds, and the public-domain ebooks I've been grabbing from Project Gutenberg and MobileRead and comics from Digital Comic Museum across multiple devices.

I'd done some command-line RSS before, and also set up backup systems with Toucan, but figured I'd try something different on this one. I gave Ubuntu One a shot and it seemed promising until I realized it isn't open-source and I can't set up my own server. Canonical is swiftly becoming the Apple of the Linux world -- good at taking open-source software and making it pretty and usable, but not so great at giving back to the open-source community.

Ultimately I settled on Unison, which proved to be a bit of a headache -- frankly if anybody has a better solution I'd be happy to hear it, but here's how I got it to work.

First of all, the Unison GUI requires GTK. Hardly a problem on the Linux side, but under Windows, extracting the binaries from gtk.org and setting the PATH variable didn't work, no matter what I did. Maybe it's a Windows 7 thing, or maybe it's a Unison thing, but either way, Unison threw up "This application has failed to start because libgtk-win32-2.0-0.dll was not found. Re-installing the application may fix this problem." every time I ran it. Sticking it directly in the GTK\bin directory worked but is an ugly solution; multiple sites suggested installing Pidgin, which comes with GTK, but produces the same problem as Unison doesn't find it in the path.

(Actually, let me back up a bit: I couldn't get Unison to work with 64-bit GTK at all. The only Unison binaries I could find were 32-bit; I opted to install a 32-bit version of GTK rather than stick Cygwin on my HTPC and compile Unison from source.)

Ultimately, I found a binary Windows installer for GTK (conveniently the first Google match for gtk windows binary installer); whatever my PATH problem was, this installer fixed it. The Unison GUI was up and running, from its own folder.

Next problem, though: SSH. Unison did not play nice with PuTTy.

Googling the problem, I found a page called Unison-ssh, which includes a wrapper named ssh.exe for download. If you've read this far you've probably already installed PuTTy, but in case you haven't, you'll only need it if you want to use public key authentication -- this ssh.exe will automatically install a copy of PuTTy's command-line SSH utility, plink.exe, if it can't find it. (Well, hypothetically. It tries to stick it in WINDIR and if you're not running it with admin privileges it'll fail.)

Now, I should add that this ssh.exe doesn't work properly under Windows 7; it'll prompt you for a username but only let you type one character and then automatically Enter it. Same problem with the password prompt. The comments thread in the page is filled with people who have the same problem. Maybe a clean compile would fix it, I don't know; again, I didn't want to go to the trouble of setting up compilers on my HTPC.

There's a solution a ways down the comments thread. Unison stores its data in the .unison directory, even under Windows. (That'd be \Users\name\.unison under Win7.) They're simple text files with the .prf extension. And you can add an "sshargs" line to give command-line arguments. If you're comfortable sticking your password in plain text, you can add the line "sshargs = -pw [pass]" and you're done. But if you're not, you can set it up with RSA keys. A later comment links a post on Palin's Technical Blog that runs down how to generate a keypair with puttygen -- the problem is, I couldn't get my Linux server to accept it; I kept getting a "Server refused our key" error.

I found the solution on Andre Molnar's blog: you need to generate the keypair on the Linux server, using ssh-keygen, add the public key to your authorized_keys file, then move the private key over to the Windows machine and use puttygen to import it and then save as a PuTTy .ppk file. From there, add "sshargs = -i [path to private key]" to the appropriate .prf file.

Almost done, but the Unison GUI still has path issues, even if you stick ssh.exe in the same directory as PuTTy and add that to your PATH. I got around it by sticking a shortcut on the desktop with the PuTTy directory as the working directory.

In summary:

  1. Install openssh-server on your Linux server and PuTTy on your Windows client.
  2. Install Unison and its dependencies on your Linux server. (It's offered in the Ubuntu repos; command-line is unison, GUI is unison-gtk.)
  3. Install Unison on the Windows client.
  4. If you want to use Unison's GUI, install GTK on Windows.
  5. Download the ssh.exe wrapper for PuTTy. Stick ssh.exe in the same directory as PuTTy and put that directory in your PATH.
  6. Generate an RSA keypair on your Linux server using ssh-keygen. By default it will put the keys in ~/.ssh/id_rsa and id_rsa.pub.
  7. Copy the contents of the public key (id_rsa.pub) to ~/.ssh/authorized_keys. Remember to set perms on ~/.ssh to 700 and authorized_keys to 600.
  8. Move the private key (id_rsa) to the Windows machine. That's move, not copy; delete it from the Linux side as you don't want to store the same private key in more than one place.
  9. Run puttygen.exe. Import your existing private key, then save the result as a new .ppk file. Delete the original key file. Again, only the owner should have read perms on this file.
  10. At a minimum, your \Users\name\.unison\foo.prf file should contain the following:

    root = [Windows path]
    root = ssh://[user]@[host]//[Linux path]
    sshargs = -i [path to private key]

  11. To get the Unison GUI to run ssh.exe properly, create a shortcut and set its working directory to the PuTTy directory.
  12. You can schedule regular syncs using Windows Task Scheduler; run the command-line Unison executable, with args "-batch [name of pref file]". Don't include path or extension, just the filename ("foo" in my example above).

So there you go: a cross-platform syncing solution. Good for backups, for keeping files consistent between your desktop and your laptop, or for anything else that requires keeping the same files on multiple machines.


Playing: Just finished playing a fan translation of Act Raiser. Maybe a bit more on that soon.

Reading: Blood of the Elves. As I await The Witcher 2.