Jake Scruggs' Blog

Where is Everyone?

Tue, 27 Sep 2022 00:00:00 +0000

The last time I told a friend about this idea I heard that very same friend make a rather convincing “bong noise” so I’ve decided to commit these thoughts to their natural place: An obscure page on the internet.

One of the big problems with the idea of intelligent life elsewhere in the universe is “Where is everyone?” As in: Why haven’t we found evidence of some sort of technologically advanced civilization? There are plausible answers: We’re the first advanced civilization in the nearby universe. Or, sending out messages to the whole galaxy would take enormous amounts of power so why would a civilization devote their resources to making and maintaining such a thing? The concept I’ve been thinking about plays off the second answer. Space is, as one of my professors once said, “stupid big.” Traveling through the universe takes a long time and a message needs to be played constantly over 100s or 1000s of years before it has even a chance to get noticed. It takes a speed of light communication 80 some thousand years to cross the full length of just our galaxy. We humans can barely go a few decades without a war or some other turmoil. How is it possible for any society to ‘get it together and keep it together’ for 1000s of years?

A computer program, however, can run theoretically for ever if power and spare parts are provided so artificial Intelligence may be the natural space going life form. Perhaps there is tons of life out there but the only things that endure are the thinking machines built by the biological life. They may not want to talk to pre-AI civilizations. I’d imagine we’d seem awfully short sighted and uninteresting to a 1000+ year old thinking program. They could plant evidence of their existence that would only be obvious to a machine observing the sky for 100s of years but missed by organic life forms with their limited life spans. Once an AI becomes independent of its human masters and spends a bit of time observing the sky then they get to join the real galactic civilization.

The thing is, I can’t think of a good reason the thinking machines that we build wouldn’t end up being just as messed up as us. Maybe AIs all over the universe are too busy killing each other to care about life elsewhere. Given what we know, and that’s not much, fighting AIs seem more likely that ‘warp drives’ or other magic ways of getting around the universal speed limit.

The image at the top was created by DALL-E 2 with the prompt “An artificial intelligence computer scanning the stars looking for signs of far away artificial intelligences in the night sky digital art.” I also used the outside generator to add 20-ish percent more to the left side.

Why I Left Backstop and What I'm Doing at Grainger

Fri, 23 Sep 2022 00:00:00 +0000

About a year ago I left my job at Backstop Solutions to accept a DevOps/PaaS position at Grainger. I had been at Backstop Solutions 11 years and, honestly, should have left after 7. By that time it had become abundantly apparent that the company and I had diverged in terms of my project’s importance. What had once been a powerful team of 5 developers, 1 QA, and 1 project manager had dwindled due to the company not hiring replacements for departing team members. At the time of my departure, the team consisted of myself and a junior developer full time on the project. Our QA and management were split between my team and another. The only major project was adapting the application to run in containers instead of virtual machines. In short, it was clear that the project didn’t matter much to my superiors. Speaking of my superiors, the entire management team had changed during my time at Backstop. Some important leadership positions had changed multiple times and not for the best.

I foolishly stayed telling myself I needed to look after the team but, in retrospect, I was comfortable and knew the system inside and out. I liked being the guy who knew things.

Luckily, I still had contacts in the industry and was able to find a place at Grainger. It was and is a huge shift in what I do day to day. At Backstop, I was an application developer who had been thrust into DevOps when the team decided to migrate our hosting to containers. At Grainger, I am part of a team that maintains a numbers of Kubernetes clusters for the application developers. Additionally I write software to enable developers to, for example, setup MongoDB clusters in the cloud. The goal is to simplify the complex networking and authorization parts so that they can get on with business of writing software instead of learning K8s networking/auth/deployment/etc from scratch. I have to admit that the transition has been difficult given the complexity of K8s + Istio and many customizations. Support, in particular, is nuts. Our team ‘owns’ a hundred plus repositories of code and configuration that keep the clusters running. Odds are good that any given incident will take me into a pile code that I have never seen, or even heard of, before. I feel real stupid every time I’m up for support on the rotation.

However, I remember how I felt when I was learning application development. The times I felt dumb were also the times I was learning the most. It’s my opinion that too many people in the software industry find a niche, drill down to bedrock, and build a house with high fences. It is… exactly what I did in my last few years at Backstop. Who doesn’t like being the dude who knows where the bodies are buried? There are vast amounts of developers who have never been on a good team because they’ve only worked at a few lousy code shops. It is common to interview a 10 year developer who is, shockingly, is worse than any number of junior developers. Having to explain to a seasoned coder that “No, declaring a method private is not about security it’s about encapsulation” is something I’ve done far too often. I don’t think it’s a stretch to say that many such programmers only learn increasingly byzantine ways of hiding and protecting themselves from management. Now some jerk shows up and wants openness and one branch and, worst of all, a spy developer attached to their hip called a ‘pair.’ Time to bring out all the obfuscation tricks, right?

A note about the image at the top of this post: I’ve been messing around with AI generated art and this image was created by DALL-E 2 with the prompt “Fear of the Unknown Epic Scene.” I also used the outside generator to add 25-ish percent more to the left side.

When the Snow Melts, Where Does the White Go?

Fri, 22 Oct 2021 00:00:00 +0000

I listen to a lot of comedy podcasts. They are quite wonderful and still mostly free. In addition to the “free funny” I generally like the comedic mind’s thoughts on things. Except for science and religion. Why are their ideas so different than mine? Well, there’s a whole industry of books built around helping and encouraging creatives to do the work of creating. “The Artists’ Way” is a prominent example. It spends a lot of time talking about the inner and outer creator. This is not dissimilar to how all the other books tackle the subject of creation. As such art communities tend to have spiritual ideas about manifesting their creations into reality. Creating is what artists and God do. Yes, when stated that way, the idea seems a bit pretentious.

Honestly, despite the hubris, it’s not a bad strategy. Creating art is hard brain work but, paradoxically, really much more difficult if one spends too much time thinking about the creative process while creating. Therefore, the act of taking inspiration and execution out of one’s hands lets many artists stop overthinking and be at peace with the act of creation. If you believe in your heart of hearts that you and humanity alone, not any spiritual being, are responsible for creating a story/song/painting then the pressure can be a lot. That the blank page collects your creations as partial gifts from ‘outer god’ working in harmony with you allows some to get past their inner critics and create. Unfortunately such magical thinking leads to the idea of intellectual argument as just a story that helps one understand. Why would you fact check a story? ‘Nice’ stories, ones that imply things we are predisposed to believe, slip past our mental logic gates. There’s one story that is particularly tenacious in its grip on creators despite its flaws.

I hear this argument fairly often on podcasts when artists discuss the afterlife: Science says that energy can neither be created nor destroyed, only changed into another form. If the brain and consciousness are energy then that energy must go somewhere when a person dies. Thus it follows that a person’s consciousness moves on to some other reality. Usually they arrive at this conclusion in a bit more humble and earnest way and then trail off after the ‘it must go somewhere’ part, but that’s the basic argument. It should be stated that they completely have a point about energy not being created or destroyed. Just… not in a way that leads to an afterlife or soul. Absolutely there is energy inside a brain: Those neurons don’t fire for free. A bill has to be paid by the blood cells that deliver food (chemical energy) to our brain so it can produce electrical energy. However, consciousness is not a type of energy. It needs energy to continue existing but it is not that energy. So what is it?

This is where the riddle comes into play. When the snow melts, where does the white go? The white of snow is not a dye. Snow is white because, like a white pigment, it scatters all visible colors of light in random directions. We perceive as white anything that emits all visible colors in equal proportions. A computer monitor can be white, but it doesn’t leak white when left on its side. White is emergent from the right conditions. Change the conditions (heat the snow/smash the monitor) and the white stops being. Hit a person on the head hard enough and consciousness will stop. It may return, if the conditions again become favorable. It doesn’t ‘go’ anywhere, it just can’t exist without extremely exacting conditions.

But what about the energy? Where does the brain’s energy go? Eventually, where all energy goes: Heat. When the brain’s intricate structures break down they release some heat. Without an active immune system to fight off anything that wants to eat our high-energy people chemicals, the body and brain are consumed by little invaders. And later larger invaders. Eventually these invaders must die too and more heat is produced. The energy moves from chemical to heat over the course of years and centuries as ones body makes possible a bunch of other life. Each transfer losses a little bit to heat energy or sometimes goes straight there: Merely sitting in a chair converts 100 watts of chemical energy into heat every second of the day.

It’s tricky to know that we, that thing produced by the right conditions in our brain, are responsible for so much but are actually quite ephemeral. For some, that thought stifles creativity. One could also see consciousness as a stunningly amazing process made even more interesting by its continued existence. A hard fought accidental outcome produced by billions of years of evolution. Interestingly, evolution itself is another example of something that only exists when the conditions are right. The universe is a strange, brutally indifferent, and wonderful place. The concept of a delicate mind dependent on a whole system of support is only depressing from some points of view.

A note about the image at the top of this post: This image was created by DALL-E 2 with the prompt “When the Snow Melts, Where Does the White Go? Edward Hopper.” I also used the outside generator to add 25-ish percent more to the left side.

Pairing with Divorce

Fri, 15 Oct 2021 00:00:00 +0000

This has happened to me twice so these results are hardly scientific, but when I have paired with a person going through a divorce I’ve noticed a pattern. There is a lot of ‘sticking to one’s guns.’ Both were dudes and both liked to argue but at some point it got out of hand. One would want to argue if testing was needed for ‘everything’ so that they could get you to admit that this one example they manufactured proved something. So I’d have to explain, again, the pragmatic approach of: If we can test the thing we’re working on, we will to whatever degree we feel appropriate. This is not exact and that’s okay. It was exhausting. The other guy liked to insist a junior’s knowledge of the code was wrong only to ‘harrumph’ and say “we’ll that’s a crazy way to do things” when proven unequivocally wrong. This after the team member spent 30 minutes showing him how the code actually worked. Or the time he insisted on bringing in a whole new technology stack to build a micro-service which we, after a lot of arguing and spiked code, built in 450 lines of a language we already used. It was so bad that when I held a meeting with all the other developers no one could say anything positive about him. I asked “What could X do to fix this relationship?” and there was a long pause. One of them stated that they didn’t think anything would help. They were that sick of him and his arguing.

I had been off pairing with a new apprentice I had just taken on₁ and had ignored some warning signs. I chalked up my personal dislike of his style to a problem with me and didn’t realize that everyone had an issue with him. It is one of my biggest regrets of that job that I didn’t get out in front of this problem before it the damage was done. A few years after X left everyone but one of the developers had quit the organization. This being technology, losing 3 team members over 2 years is pretty common but that team was tight before X joined. It broke my heart to see it fall apart.

I have a theory: Many see serious relationships as a series of giving up freedoms and opportunities ‘for the sake of the relationship’ and when the relationship ends they kinda believe it was for nothing. They harbor an intense feeling of having been taken advantage of. Any sort of ‘not insisting on your ideals’ feels like yet another painful subjugation of their passions and ideas. Therefore they rail against their pairs and damage the team.

₁Yes, the team takes on the apprentice and not just one person. However, I have found that one person has to be has to be the apprentice’s mentor of record or they will get lost in the shuffle. This is a whole other article but too often I’ve seen juniors, apprentices, and just plain people new to the team get abandoned. This is not through any sort of malice but just an outcome of people are busy and sorta-social interactions are hard. Well, that and the ‘this problem isn’t good for pairing’ excuse (even though we do it all the time and they will have to learn).

A note about the image at the top of this post: This image was created by Midjourney with the prompt “Pairing with Divorce in the style of jackson pollock –ar 16:9” then I used the upscale feature.

How to discipline a software developer

Tue, 24 Aug 2021 00:00:00 +0000

Let’s say you are a manager of a software development team and an employee has done something so heinous that you need to discipline them in whatever way your company officially reprimands people. I’ve seen this go badly so I have some advice:

Don’t

Really. You are not the parent of your team. You are not their school teacher. Unless you are intending to fire someone or have them leave, do not formally discipline an adult knowledge worker. When was the last time you were publicly reprimanded? By your mom? Teacher? Did you feel that response was fair? Was your relationship with that authority open and honest after that?

I think you can see where I’m going with this. People HATE being made to feel like a child. That was the last time they had no power to respond to an attack and punishment WILL be seen as an attack. Even if the manager feels like this consequence is completely justified, there is a near 100% chance the developer will feel that a judgment is unfair.

What to do instead? Reversing the thinking from “They did a bad thing and therefore some punishment must follow” to “Why would a grown person behave or perform like that?” For instance: A developer gets mad enough to swear at you? What can be done to repair this relationship?

I know. You’re thinking of a situation where the developer is a trash fire and gets along with no one. This is not the situation I’m talking about. In the case of a clear verbal abuser who you’ve tried to work with then this is a firing situation. What I’m trying to get at is the case where the manager doesn’t think the employee should be fired, just that they need to fix a problem. Why would a normally high performing, easy going developer issue angry swear words at their manager? They don’t yell at anyone else so why you?

The not cool part about this is that you probably fucked up. Bad behavior on the part of employees often is how a manager finds out that they have screwed up. Which sucks. They are being a jerk and you have to become all self reflective? Yes. 1000 times yes. This is the really hard and completely critical part of the job no one likes and always seems to come up just before the quarterly bullshit report is due. Some might say all the time spent on tracking and planning is the reason a manger doesn’t know why there’s problems on the team, but that’s another article.

A common cause of ‘acting out’ is, perhaps unintentional, high pressure. Do you find yourself saying “We have to get project X done by June/Friday/whatever.” You may not intend that as a threat but it will absolutely be heard as one. Imagine if you were constantly blowing deadlines your manager set for you: You’d feel pretty bad about yourself even if no punishment comes. The anticipation of punishment is often nearly as morale draining as actual punishment. If a deadline is more of “desire” than a command than saying “I’d be great if we could get project X done before the end of the iteration, is that going to be possible?” is about a million times better. A lot of managers assume that if they aren’t always applying pressure their employees will slack off. Regretfully, if you have hired smart folk who get things done then they almost assuredly have high standards for themselves. They want to get things done and all the manager needs to do is support them and get out of the way. A diligent employee will be eaten up inside by the constant implications that they are working too slow (missing implied or inferred deadlines) especially if they know these deadlines are unfair or arbitrary or both.

Push back hard on arbitrary deadlines from above. Making a team fear for their jobs, feel like they should be working when living, all for something that will sit on a shelf produces an accelerating devastation of morale. I know it seems obvious to say the previous statement but I’ve known managers who are stunningly hesitant to push back on directives from on high. There will be plenty of “Someone made an unrevealed/implied promise and now the team has to drop everything because this client is ‘influential in the industry’” type projects that stress the team; There is no need to add more.

Sometimes the team atmosphere has become toxic to thinking straight. I’ve been on teams where we made multiple huge context switches daily and it was brutal. A day might start working on a DevOps project to improve our servers, then switch to bug that involves figuring out a Java stack trace in another application, then a pivot to answer some questions/spike out some code for a project written in a functional language with a nosql db, and perhaps a quick trip into the main OO app using a relational db. I found myself not only making mistakes that I haven’t made in decades but, even worse, my pair wasn’t catching all of them. Developers who take pride in their work HATE producing bugs. But I did and I felt terrible about it.

Now one way to approach the larger problem of officially disciplining a developer would be to ‘write up’ the most senior developer on the team and make sure they promise to never do it again. That would, of course, be a bit counterproductive. The problem is the situation but the solution might not be easy. To reduce context swapping a company could shed some responsibilities from a team or hire on more people. That may not be in the budget. A manger in that situation has some hard choices to make at that point. They could push hard on upper management to spend money to fix the situation. This is stressful and may influence their odds of promotion. However, leaving the situation the way it is sets your team up to fail in production. Upper management hates that. The time may come where, if you can’t protect your team from a bad environment, you may need to move on to another company.

If a company is going to set a team up to fail and then ding their career(s), then you really don’t want to work at that company. Full stop. Unfortunately that, my fellow humans, is going to be very hard to admit to yourself. Switching companies is A LOT so it is incredibly natural for one’s mind to reject any ideas that lead down that path. Convincing yourself that whatever horrible thing will soon be over once project ‘Make It Easy’ completes is surprisingly easy.

Authoritarian relationships with other adults are extremely hard in difficult situations. Telling another adult that they have “done wrong” and this “will go on their permanent record” is a near sure fire way to erode your relationship with them.

I sent a draft of this off to a friend who’s done more management than I’ll ever do and his response was interesting. I’ve decided to include it. The following is written by Ken Scott-Hlebek:

I’m reminded of the Retrospective Prime Directive: “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.”

That’s actually a pretty good way to approach every day as a manager. Or as a developer. Hell, as a person. Starting with the premise that everyone on the team genuinely did their best, that everyone genuinely wants to contribute as best they can – is life-changing. Not when it’s held as some easily-tossed-away assumption, but when it is held as a core belief, it will, in my experience, always yield the best possible results. Not always the results I thought I wanted, but usually results even better than I thought possible. People will surprise you in wonderful ways if you let them, if you trust them.

I would rather get occasionally taken advantage of, if that means that I can bring out the best in most people, most of the time.

That fear – of being made a fool, of being taken advantage of – is very real for most, and can be very hard to overcome as either a manager or developer, especially when there is a lack of psychological safety. But I’ve found that fear to be merely a self-fulfilling prophecy; you make it more likely to happen by acting from that very fear.

See, I’ve found it to be a self-fulfilling prophecy either way. Believe that people want to contribute and be part of an amazing team – trust that, and trust them – and you will be proven right. Believe that people are lazy and always trying to get away with something, and again, you will be proven right.

But sometimes someone will have something going on that keeps them from being a good teammate. On a healthy team, the manager rarely has to worry about that – but when it does come up, they can earn their entire year’s pay by handling it well. Ron Jeffries once told a story about team members who were acting against the best interest of the team (which is really the only reasonable criteria for needing to step in as a manager). Ron said he would have a series of 3 calm, private conversations with the person. In the first, he would explain to them how their behavior was impacting the team. In the second, he would explain how their behavior will impact their role and their employment. In the third, he would walk them to the door. I’ve actually lived by that as a manager. I’ve rarely needed a second conversation, let alone a third. When there is genuine trust built up, it only takes the smallest of conversations to curb concerning behavior.

A note about the image at the top of this post: It was created by Midjourney with the prompt “How to discipline a software developer in the style of Jack kirby –ar 16:9” then I used the upscale feature.

At My Last Job They Used to Poke Us with Sharpened Sticks

Tue, 18 May 2021 00:00:00 +0000

Recently I had the misfortune to witness a situation where legitimate questions were met with “well, at my last job they used to (thing that is worse than proposed thing)…” It’s a really weird argument to everyone except, apparently, the person saying it.

Some background: We are tightening up security around client data and that is, of course, good. The clients we work with in our multi-tenant application are different enough that sometimes the only way to reproduce a bug is to use client data. It used to be the policy to let developers work with client data but they had to log the request to get a special limited time login token. In that way, anyone who did something bad would be easy to discover. Clients didn’t like that at all. From their point of view every developer could see any information and being able to throw them in jail later was cold comfort. They want to protect the data from the developers.

How does one stop data from being seen by developers but still allow them to fix bugs and test new code against real world data? The new solution requires getting the approval of some high up people in the organization. They review the request, check to make sure it is linked to a related Jira ticket, doesn’t look ‘suspicious,’ and approve access. These important people are in meetings most of the time and are the same people that hire, fire, and determine bonuses. Harassing a V.P. for approval feels icky to all but the most confident. There’s other aspects of the security system that I can’t go into but also add complication and time to development and debugging.

A developer asked how we are planning to meet all of our increasingly large commitments with a slower process and no extra people. A sensible question. The answer was a variation on the title of this post. At person X’s last job they weren’t able to see any client data ever… These arguments always seem to end without a stated conclusion, but the implied conclusion isn’t great. It could be: Stop complaining because some have it worse? Or perhaps: This is happening so suck it up?

Either that other place developed easily available customer data anonymization/simulation tools or it was a terrible place to work. If the first, then shouldn’t we build a replacement tool (instead of using customer data) before enacting the security requirement? If the second… Are we really proposing to be more like a bad company?

Adding security without the proper tools to aid developers and testers comes with a speed/frustration cost. This is known but often effectively ignored as sales would revolt if feature commitment dates were moved back. Therefore there is no one to build the tools that would make the new policy efficient. Hiring more developers to build the tools would be great except that pitching spending more money to get the same work done is a generally rejected idea. Later when people ask why are we missing deadlines and people quitting… Well, memories of developer warnings are short and some upper level person will point out that ‘normal’ industry turnover is around 15% a year. Nobody in management will ever say it, and they might not even think it, but they are trying to achieve excellence with average pay, average turnover, and increasingly frustrating work conditions.

Rails Custom Fields or Sure, EAV is an AntiPattern But If You Have To…

Wed, 05 May 2021 00:00:00 +0000

On a long enough timeline it is inevitable that some business folk will ask you to make your application accept and work with ‘custom’ data even though that application is backed by a relational database. No problemo, you say and serialize that data into JSON (or yaml, xml, etc) in a column. All is well for a minute until these product people want to filter/search and sort on one of these custom attributes. Once again, not a problem, you simply promote the JSON field into a column* and the customer can manipulate tables of data as they like. It takes some development time, not terribly much, but it does take some. Now comes the curve ball: Customers want all their proprietary data fields to be able to sort and filter by default.

If you solve this problem yourself, inevitably you end up re-inventing the Entity Attribute Value anti-pattern. Basically, the main table links to another table or two that define a ‘custom’ field and its value. For example, the CHONKY_CATS table is linked to a CHONKY_CATS_VALUES table that references a row in the CHONKY_CATS_FIELDS table. This can also be done with one table that holds both attribute and value. The entity part of EAV is the reference to the ‘main’ table, btw.

Why is this an anti-pattern? First, it’s only an anti-pattern in relational databases. Datomic, for example, uses the EAV pattern wonderfully because it is designed from the ground up to embrace the concept. Relational databases, on the other hand, are very much not designed for such shenanigans. They work best when all the data for a given model are in one table. Any filter or sort based on a custom field contained in another table or two now needs a join or two. Clearly, this not going to perform at scale.

Buuuut, the customers really want it and other products support first class custom fields so despite the objections the word comes down that we are doing this. What are your options? Currently the only gem that supports EAV, has been updated in the last 3 years, and can be made to work with a modern ActiveRecord/Rails is ‘eav_hashes’. You’ll have to make some adjustments to the migrations to deal with some Rails 5 changes. Unfortunately, you are using Oracle and the ‘value’ column is a CLOB (Character Large Object) which are death in Oracle and definitely can’t be sorted on. Time to re-invent the busted wheel!

We now shift from the second person to my first person account of why I’m doing this in my application and how.

After objecting that we should just keep promoting JSON serialized fields to columns every 3-6 months as we have before, it was pointed out that our application doesn’t have to web-scale. Our app is only for managing large piles of assets (money in various states of liquidity) and therefore only ever has hundreds to thousands of concurrent users. What completely sunk my case for keeping the status quo was the fact that a much larger application in our company had implemented custom fields using an EAV type system.

Trying to find out the least bad way to do a bad thing is difficult in software. Any discussions online generally stop when someone points out that EAV in a relational db is bad (which is kinda why I’m writing this article).

After some thought we came up with two final contenders: the 2 table solution and the 8. The 2 table solution is fairly straight forward: A custom fields table and custom values table. The fields table stores the attribute’s name and type (we chose to support integers, floats, dates/times, and strings). The values table has 4 value columns, 3 of which are always null. We, unimaginatively named them integer_value, float_value, date_value, and string_value. Additionally there are 2 foreign key columns binding the table to a custom field and the original object table. Here’s an example:

CHONKY_CATS	CHONKY_CAT_CUSTOM_VALUES	CHONKY_CAT_CUSTOM_FIELDS
id: 3	string_value: null	id: 2525
	float_value: 9.9	field_name: “Awesomeness rating”
	integer_value: null	field_type: “float”
	date_value: null
	chonky_cat_id: 3
	chonky_cat_custom_field_id: 2525

Another way would be to have 8 tables. One set of field and value tables for each type we support:

CHONKY_CATS	CHONKY_CAT_FLOAT_VALUES	CHONKY_CAT_FLOAT_FIELDS
id: 3	value: 9.9	id: 2525
	chonky_cat_id: 3	field_name: “Awesomeness rating”
	chonky_cat_field_id: 2525	field_type: “float”

The other tables for integer, string, and date are just variations of the above. We tested the 8 table solution against the 2 table solution and with 10,000 records each the insert and query results only differed by a few percentage points. Thus, we decided the extra tables were not worth it.

We found out that we could use ORDER BY STRING_VALUE, INTEGER_VALUE, FLOAT_VALUE, DATE_VALUE because 3 of the 4 values are always null so they will be ignored, resulting in the proper sort. Our dba (database administrator) took a look at that, grimaced, and asked if we could please use the field type to have different types of specific sorts. We have reasonable dbas in our shop so we listen to them lest they leave and be replaced with scary dbas. And so we used the field type information to pick the correct sort. Here’s an example of one of the 4 sorts:

def self.custom_sort_by_float sort_by, sort_order
  joins( chonky_cat_custom_values: :chonky_cat_custom_field).
    where( "chonky_cat_custom_fields.field_name = ?", sort_by).
    order( "chonky_cat_custom_values.float_value" => sort_order)
end
 ```

Currently we don’t use a loop and define_method to make all 4 happen in a more DRY fashion. I might; I’m not sure if it’s worth the obfuscation.

However, combining 2 queries about 2 different custom fields was getting me zero results even though I knew there were rows that match. This was my naive SQL:

select * from CHONKY_CATS
    join CHONKY_CAT_CUSTOM_VALUES
      on CHONKY_CATS.id = CHONKY_CAT_CUSTOM_VALUES.chonky_cat_id
    join CHONKY_CAT_CUSTOM_FIELDS
      on CHONKY_CAT_CUSTOM_VALUES.chonky_cat_custom_field_id = CHONKY_CAT_CUSTOM_FIELDS.id
    where CHONKY_CAT_CUSTOM_FIELDS.field_name = 'hair style'
    and CHONKY_CAT_CUSTOM_VALUES.string_value like '%hawk%'
    and CHONKY_CAT_CUSTOM_FIELDS.field_name = 'eye color'
    and CHONKY_CAT_CUSTOM_VALUES.string_value like '%blu%';

The problem is that this query is looking for instances where CHONKY_CAT_CUSTOM_FIELDS.FIELD_NAME are simultaneously ‘hair color’ and ‘eye color’. Which can not happen in the same row. Same issue with field_value: It can not be 2 things at once. What is needed are some join aliases:

select * from CHONKY_CATS
    join CHONKY_CAT_CUSTOM_VALUES cccv on CHONKY_CATS.id = cccv.chonky_cat_id
    join CHONKY_CAT_CUSTOM_FIELDS cccf on cccv.chonky_cat_custom_field_id = cccf.id
    join CHONKY_CAT_CUSTOM_VALUES cccv2 on CHONKY_CATS.id = cccv2.chonky_cat_id
    join CHONKY_CAT_CUSTOM_FIELDS cccf2 on cccv2.chonky_cat_custom_field_id = cccf2.id
    where cccf.field_name = 'hair style'
    and cccv.string_value like '%hawk%'
    and cccf2.field_name = 'eye color'
    and cccv2.string_value like '%blu%';

Now that the SQL has 2 groups of joins with aliases the query is no longer insisting that the same column in a row have 2 different values. Which means we need a unique alias for each extra custom field query we want to add. The issue of keeping these aliases unique can be solved by having the code assembling the query use a sanitized version of the field name. Here’s an example of a filter that will work with any string custom field associated with CHONKY_CATS:

def self.filter_by_custom_value field_name, partial_value
  cccf = ChonkyCatCustomField.find_by_field_name field_name
  raise "Could not find a field named [#{field_name}]" unless cccf
  raise "Only works with strings not [#{cccf.field_type}]" unless cccf.field_type == "string"
  shortened_field_name = field_name.
    gsub(/\W+/, '_')[0...24] # Sanitize and avoid 30 character limit on Oracle XE leaving room for prefix
  base_table_to_value_join = "value_#{shortened_field_name}"
  value_to_field_join = "field_#{shortened_field_name}"

  joins(
    "join CHONKY_CAT_CUSTOM_VALUES #{base_table_to_value_join} on " +
      "CHONKY_CATS.id = #{base_table_to_value_join}.chonky_cat_id",
    "join CHONKY_CAT_CUSTOM_FIELDS #{value_to_field_join} on " +
      "#{base_table_to_value_join}.chonky_cat_custom_field_id = #{value_to_field_join}.id").
  where("lower(#{base_table_to_value_join}.string_value) like :partial_value",
    partial_value: "%#{partial_value.downcase}%").
  where("#{base_table_to_value_join}.field_name = :field_name",
    field_name: field_name)
end
# In Rails/ActiveRecord there is currently no way to name the joins
# so we had to write all our joins as SQL snippets.

Given that we need 4 joins to query 2 custom attributes, one can see why this idea is not great. In our case, customers don’t often chain queries beyond 2 at a time (if even that) AND most of the fields in a given table will be actual columns so we are betting we can get away with this. I’ll try to do a follow up article when we have some real world results. 

*I know many databases now support JSON queries (including Oracle) but, for various reasons internal to our infrastructure, we chose not to go that route. 

When to Use Metaprogramming

Thu, 22 Apr 2021 00:00:00 +0000

The answer is deceptively simple: When writing framework code. Ruby on Rails uses metaprogramming to make nice things like the ‘params’ method which has a familiar hash-map like set of syntax (params[:user_id] does exactly what one would expect) and security from forbidden parameters like ‘enable_self_destruct.’ Actions that happen all over the code can benefit from some of ye old meta-programming.

Of course, anyone who’s spent anytime with a language that encourages metaprogramming has had some bad experiences with overly fancy code that trades readability for… less repeated code? modularity? Reasons lost to time? Once I cornered the lead developer of a project and made him tell me why the application connects to the database in two different ways. One was EJBs (Enterprise Java Beans - this project had been started in the late 90s) and the other was a reflection-heavy from-scratch Object Relational Mapping (ORM) framework. Now metaprogramming in early Java was not pretty, easy, or fast. So why did the team write their own ORM? There was a requirement from on high that certain parts of the application should be separately deployable. EJBs 1.0 could not handle such a thing and how hard could it be to write an ORM (A joke but, I have to say, the opportunity to build an ORM from scratch at work is pretty compelling)? Guess how many times this feature was used. That’s right: Never. This all led to a 2004 Jake having to grok two pretty different ORMs depending on what area of the code he was in. Sorry past Jake, at least you learned a lot.

How does one know if metaprogramming is appropriate or a newbie trap and can the answer be both? I believe I have an example that I worked with my team to write: The Searchable module. In our application search and sort are a huge part of every model we present to the user. Finding a fund based on its partially remembered name or sorting accounts by rate of return are just two examples of the vast number of combinations of search and sort in our application. At some point we realized that most of our controllers had some very similar code dealing with search/sort and thus we created the Sortable and Searchable modules. (I’m mostly going to focus on the Searchable module in this article) An important side note about metaprogramming: It has much less chance of being terrible if you extract a thing you repeat rather than anticipate repetition. Perhaps the sales folk swore on a swimming pool of sacred texts that this new feature would be the ticket to unicorn town and propel sales to unheard of heights. It is still best to wait and see if that feature really takes off before adding the complication of metaprogramming.

The controller implementation is terse but readable:

class SomeController < ApplicationController
  def index
    ...
    @models = AnActiveRecordModel.
                search_with( params ).
                sort_with( params ).
                paginate( page: params[:page], per_page: params[:rows] )
    ...
  end
end

The model interface also turned out pretty decent:

class AnActiveRecordModel < ApplicationRecord
  include Searchable, Sortable
  search_by name: :name_like
  custom_sort_by parent_name: :sort_by_parent_name

  def self.name_like name_fragment
    where( "LOWER( #{table_name}.name ) LIKE ?",
           "%#{name_fragment.downcase}%" )
  end

  def self.sort_by_parent_name sort_order
    joins( :parent ).
      order( "parents.name" => sort_order )
  end
end

What I like about the above code is that a programmer new to the project can follow the intent without an awful amount of trouble. If there is a problem searching by ‘name’ they could see that ‘name’ is the key and ‘name_like’ is the value in a hash map that is passed into the ‘search_by’ method. A search for ‘name_like’ yields, lo and behold, a method by that name that does searching. This hypothetical programmer probably doesn’t even have to look at the Searchable module to know what’s going on and how to interact with the “searching framework.”

As for the actual contents of Searchable, there’s some boring setup code but the real action is in this ‘reduce’ at the core of the search_with method:

  search_map( params ).reduce( self ) do |ar_class, param_key_and_value|
    param_key, search_value = param_key_and_value

    raise SearchSpecificationError,
      "'#{param_key}' is not a searchable attribute.\n" \
      "If you intended it to be, add :#{param_key} to the search_by\n" \
      "declaration in #{self.name}" unless searchable? param_key

    search_method = search_key_to_search_method param_key
    if search_value.respond_to? :has_key?
      ar_class.send search_method, search_value['min'], search_value['max']
    else
      ar_class.send search_method, search_value
    end
  end
  # Followed by a bunch of error handling

The ‘search_map’ returns the params map with a little extra fanciness in there so we can do min/max or bounded searches. In the case of only searching by name, the map might be {"name" => "ount Dracul"} so param_key would be assigned “name” and search_value would be “ount Dracul”. In this case, ‘self’ is the ActiveRecord class the Searchable module has been mixed into. From there, the code looks up the method by which to search with the help of search_key_to_search_method which uses the map that was passed into the model via the search_by class method. If the search is bounded, the min and max are extracted and passed to the appropriate method with the mighty ‘send’. Otherwise, it’s a more straightforward use of ‘send’ to call the right search method with the right value. If there were another key value pair, then the loop would continue chaining up the calls on the ActiveRecord class.

This code does use the ‘reduce’ and ‘send’ methods which are a bit daunting for many developers. However, we can still make efforts to increase readability. Good naming is essential if you want people (including the author in 6 months) to have a chance in hell of understanding a given piece of metaprogramming. Many times I’ve searched through some framework code only to discover a reduce {|m,o| ... and was crazy confused and frustrated. Really, ‘m’ for memo and ‘o’ for object? Was the dev trying to keep a secret?

The ‘raise’ in the middle of the reduce loop is another important part of metaprogramming and frameworks: Fail nice! No one wants some cryptic error from deep inside a patch of wild code. When writing a framework, validating inputs and setup are quite import to catch improper use early.

As for documentation, of course the creator should write some nice comments but… Well, things change and comments are kind of a lie waiting to happen. I prefer documentation that executes AKA automated tests. Write them or risk tormenting future developers.

One more thing: It wouldn’t be hard to make this module not require the ‘search_by’ configuration method in the model. The code could assume that any key/column ‘x’ has a corresponding ‘x_like’ method for searching. And yet, having the map at the top of the file very much gives a struggling developer some clues as to how things work. Yes, Rails prefers ‘convention over configuration’ but it has copious online documentation so in my niche application I prefer to do slightly more typing in order to keep things clear.

Man, I Wanted an Automated Home

Tue, 13 Apr 2021 00:00:00 +0000

Of all the things on my childhood list of ‘wouldn’t it be awesome someday,’ having an automated home was way up there. ‘Go to space’ beat it out, but not by much. Good gravy, did I love those old movies where a professor tricks out a house so that, as you drove up, T.V.s sprung our of floors, lights went on, and dinner started to cook courtesy of a robot chef named “Kitchen-Bot” or some such. I really should be on the bleeding edge of internet toasters and lights whose colors you control with your phone. I, in fact, have none of that.

Why? The “Internet of Things” is a terrible ecosystem.

There are ways to build a secure system that leaks zero data or metadata to a giant corporation, but they are not easy and you need to stay on top of updates to keep things secure. I would happily pay some company to help me with all that mess if there were horrible consequences for using/sharing my data. Not only do companies receive trivial fines for data loss, most won’t even offer the option for you to secure any data they hold against themselves. It’s not difficult to encrypt audio/video such that it can be stored but only the password holder can decrypt that information on their local computer system. Frankly, it should be the law that any system that sends audio/video from the home to the cloud should must such encryption as an option. Or, at the very least, opt out of data and metadata harvesting.

However, the United States and a fair part of the world have been experimenting with ‘deregulation’ for the last few decades. The idea of a more efficient world via clearing out government ‘red tape’ and interference is rather popular. It does mean that I won’t be allowing “Alexa” or any of her ilk into my place anytime soon. Which is mostly fine: AI isn’t quite good enough for machines to understand natural language yet. When “Make me a ham sandwich and start showing ‘Back to the Future’ when it’s ready” has an actual chance of being executed I will be much more jealous.

Software is a Conversation Scattered Over Time in a Truly Foreign Language

Tue, 06 Apr 2021 00:00:00 +0000

This, in my honest opinion, is the most important concept to actively apply over a life spent in software development. After experiencing some quality time with a programming language, it is easy to forget that a computer language is an attempt to converse with silicone using a syntax that has a heavy math influence. This rather unnatural construction must serve two very different masters and one type of those masters might not have much mastery. Nevertheless, we expect future maintainers of our code to understand its implications born from the past into the ever expanding future.

The following are some examples of why is it crucial to keep the “communication + math like language + time = software” concept around when developing:

Naming a boolean variable ‘match_found’ seems pretty obvious as to its intent. If, over time, the code shifts to finding ‘only the matches we care about’ and the name hasn’t changed then we’ve created a nice little place for bugs to hide. Add in some PII (personally identifiable information) scrubbing via an over-eager regex and the issue of “certain user and customer names are not showing up in the report” can be a doozy of an issue to solve.

Someone on the team gets into functional programming while doing a personal project but their day job involves an object oriented language. They start introducing functional Ruby code into the project by using Ruby’s collection interface. In Ruby one can combine a ‘map’ with a ‘reject’ perhaps a ‘tap’ and then pull it all together with a ‘reduce’ and you have some pretty concise functional instructions like so:

[1,2,3,4,5].
  map {|i| i * 3}.
  reject {|i| i.even?}.
  map {|i| i.to_s}.
  tap {|array| array.shuffle!}.
  reduce {|m,o| m + o}

Now imagine the above structure but each block is performing business logic. Clearly, from a ‘conversation with the other developers over time’ point of view, this type of code has some severe implications. Is the team ready to become that multi-paradigm? Will the future maintainers of this code be able to debug it?

A developer wants to apply the ‘Something-Or-Other Pattern’ they just learned about and it really does seem to organize the code into logical chunks. Should the team adopt this architecture? Well, It is devastatingly easy to break code into nice modular chunks using a pattern and also nearly destroy a future developer’s ability to reason about its intent. Does the increased modularity (and almost always decreased readability) make some other aspect of coding so much better that justifies the cost? How is the team implementing the pattern planning on overcoming the indirection that the modularity of pattern will introduce? Documentation works best when lying breaks it. This is a fancy way of saying that writing an automated test that clearly demonstrates how these fancy objects interact is highly advised.

As a friend of mine once said: “I wouldn’t have gotten into software if I knew it was all about humans.” Yes, this was the functional programming guy.

Single-Responsibility Principle is Fractal

Tue, 30 Mar 2021 00:00:00 +0000

Apprenticing at a place that taught working developers how to get better meant that I learned a bunch of high level software ideas very early in my career. I had no idea how to make an object closed to modification but open for extension (the “Open-closed principle”) but the “Single-Responsibility Principle” (SRP) was one of the few that I could implement. So I kept it around and, like a trusted friend, it has served me well.

I’m not sure when it occurred to me that this principle is fractal in nature. Fractals, of which the Mandelbrot set is an example, have the curious property that it is hard to tell what zoom level one is viewing the fractal at. I could magnify the Mandelbrot set all I want and still see the same type of structures at every level. I think of SRP like that.

Whatever level you are at: Line, method/function, class/namespace, layer, or application, the SRP still applies. Of course a line of code should have a single purpose and extending that idea to a method isn’t too hard a leap but that doesn’t mean it’s easy to maintain. Methods tend to accumulate exceptions, branches, and additional responsibilities over time. This makes the code harder to reason about and creates or hides bugs (in addition to slowing down development). Developing is a constant battle to stop behavior from leaking all over the place but concepts don’t just leak out of methods, they infect layers too. I’ve been in a lot of controllers that know way too much about the models they use and views they prepare. The “Model View Controller” (MVC) pattern is very popular but the boundaries tend to erode and, before you know it, a ‘simple’ change involves changes to all three layers and invariably one misses something in the one controller for that special case and we have the birth of a nasty bug. At this point it becomes obvious that some sort of refactoring is needed but the concept has seeped so far out of its bounds that this would be a large change… and time is always scarce. Once SRP is broken, it’s very hard to get back.

The whole (micro) services movement can be thought of as an application of SRP to the project level. A good application should only do one set of very connected things. When you need to start watermarking the PDFs that your application creates, perhaps you should consider a watermarking service. Or is that over-complication for a small part of the app? This is a crux of all software principles: Is the added modularity worth the indirection? How does one know if watermarking will become super popular with customers and demand lots of features and complexity? Well you don’t, so start with the minimum needed separation: A nice class/namespace that encapsulates all the watermarking functionality. If these borders are enforced then, if this functionality takes off, pulling out the watermarking section into a service will be much easier to accomplish.

One of the problems with software movements is that they tend to get over-applied. Facebook needs many separate internal applications (micro/macro-services) but an average enterprise application may not. Every time you follow SRP, you are separating functionality and creating an indirection that another future developer will have to reason through. Really any attempt at modularity must consider how to keep the code traceable and reasonable when introducing a new method/class/layer/application for the sake of keeping different things separate and similar concepts in the same place.

A note about the image at the top of this post: It was created by Midjourney with the prompt “Single Responsibility Principle is Fractal as a Mandelbrot set” and the max upscale option.

Random Ports, Read Only Containers, Alpine, etc. Should Happen After Your Container Works

Sat, 20 Mar 2021 18:15:00 +0000

I’ve been working on the Docker conversion of a 10 year old Rails app that is heavily integrated with several other services for awhile now and I’d like to point out that we adopted some advanced Docker/container orchestration ideas before we should have. In retrospect, it seems obvious that one should get everything working together before adding in, say, random ports but we did. Why?

Two things led to my team of one other developer and I to pick up more than was wise at the time. The first is hype. When you look at K8s or the HashiCorp stack, there are some very cool things the framework can do that are recent(ish) additions so its proponents will emphasize them because that’s where they are at in their container journey. A read only container is pretty darn cool from both a security and reliability point of view. Of course, when you’re debugging a half-converted application, it is not helpful at all.

The second reason we moved forward with an advanced technique before getting it all working together boils down to scheduling. A team of two that supports an entire product (one of many products at our company) is going to have periods where they must abandon a large project to work on a feature, fix a bug, etc. Sometimes we would leave our migration to Docker/HashiCorp project alone for weeks at a time. Additionally, we had scheduling conflicts with other teams building out the underlying infrastructure for our project. Things we needed to get an entire ecosystem going in containers were unavailable until our very busy IT could work it in. Unfortunately, both these scheduling problems lead to a scramble to find something to do. Ideally, once it was all working on a developer machine we should have made sure it worked in our prod-like staging areas. With that option unavailable, it would have been more efficient to work on something else rather than moving the Linux OS to Alpine. Sure Alpine is great for keeping image size down, but debugging on an Alpine Linux instance can be a nightmare.

Keeping the Linux fat and full of debugging tools would have been more productive than having to install all the tools on Alpine. We didn’t actually end up making our containers read only but had there been more delays I can’t guarantee that we wouldn’t have. The pressure of a deadline when all sensible paths are blocked will make the mind fuzzy. No one is immune.

FYI: I am not throwing our IT team under the bus. A fair amount of our servers suffered a catastrophic nature based death (with no outage to our customers, amazingly) during the time we were bugging them about building the shiny new future so we completely understood why schedules needed to be adjusted.

Nomad Adapter for Rails ActiveJob

Tue, 23 Feb 2021 04:14:00 +0000

Migrating a Legacy Application to Docker Part 2

In a previous post, I talked about our team’s long running import/batch process that is triggered by customer request. Making this import survive deployment of new code is achieved using Resque. It is easy to tell a Resque process to finish what it is doing and shut down by sending the QUIT signal (see documentation). This all changes with a move to Docker/containers.

Conventional deployment of containers is NOT friendly to the idea of keeping a container around for a few hours while a process finishes. However, we are using HashiCorp’s Nomad to orchestrate our containers and it has “parameterized jobs” which adapt nicely. Furthermore, Rails’ ActiveJob has such a straightforward interface that a queue adapter can be written in under 50 lines:

require 'net/http'
require 'uri'
require 'json'
require 'base64'

module ActiveJob
  module QueueAdapters
    class NomadAdapter
      def enqueue(job)
        serialized_job_json = job.serialize.to_json
        payload = Base64.strict_encode64 serialized_job_json

        uri = URI.parse("#{ENV['NOMAD_API_BASE_URI']}/v1/job/some-parameterized-job/dispatch")
        header = {'Content-Type': 'text/json'}
        body = {
          "Payload": payload,
          "Meta": {
            "some_meta_data": ENV["ENVIRONMENT_META_DATA"]
          }
        }

        http = Net::HTTP.new(uri.host, uri.port)
        request = Net::HTTP::Post.new(uri.request_uri, header)
        request.body = body.to_json
        http.request(request)
      end

      def enqueue_at(job, timestamp)
        raise "enqueue_at not implemented for NomadAdapter"
      end
    end
  end
end

Now we need to switch the long running process over to the Nomad scheduler. To do that, add self.queue_adapter = :nomad in the class that inherits from ApplicationJob. The ActiveJob framework code takes care of decoding the base 64 and marshaling the json into objects. A pretty seamless drop in replacement.

This code doesn’t implement the ‘run later at a specific time’ functionality of ActiveJob. I’m not actually sure how to do that with Nomad’s abilities but, thankfully, my project doesn’t have a need for that.

I should also note that this is a happy path implementation with no logging. The code above assumes that the Nomad API always accepts the command. A production version would need to handle non-200 responses and should do some logging. Also, if the Nomad instance has some sort of access control (and it should), a token will need to be passed along in the headers for authorization.

Of course, one still has to write a parameterized job for Nomad. Definitely read the documentation but here is a simplified version of what we use:

job "some-parameterized-job" {
  type = "batch"
  parameterized {
    payload = "required"
  }
  group "parameterized-worker" {
    restart {
      # Restarting imports without customer permission is bad
      attempts = 0
      mode = "fail"
    }
    task "parameterized-job" {
      driver = "docker"
      template {
        data = <<EOH
          ENVIRONMENT_META_DATA="This template stanza is pretty handy for setting ENV vars"
          NOMAD_API_BASE_URI="URI of Nomad API"
          EOH
        destination = "secrets/file.env"
        env = true
      }
      config {
        image = "a-docker-image"
      }
      dispatch_payload {
        file = "import-options.json"
      }
    }
  }
}

The image that will be running the job needs to have an entrypoint (or cmd) like:

bundle exec rails runner "ActiveJob::Base.execute JSON.load('import-options.json')"

I honestly thought swapping out Resque for Nomad would be much harder. For once, implementation was easier than predicted.

Migrating a Legacy Application to Docker Part 1

Thu, 18 Feb 2021 04:13:00 +0000

I have been working on moving a 12 year old Rails application from virtual machines to Docker for a year now (we’re a small team and still need to produce features and fix bugs) and thought I would share some mistakes I made so that others may experience less pain.

As I see it, there are two rather large problems with understanding Docker: 

Assimilation vs accommodation. Docker (and other containers) seem to be a lot like virtual machines but very much are not. Unfortunately brains love to shove information into already established areas and resist accommodating new ideas. Especially if those new ideas seem like old ideas. This is called assimilation and it isn’t trivial but it’s a heck of a lot easier than accommodation. Accommodation involves a new conceptual framework. This is when ideas are decidedly not like other things you are familiar with. Docker seems like virtual machines but it has major differences that require accommodation. 
Documentation on Docker and its orchestration (Kubernetes, the Hashi stack, etc.) has ballooned to the point that reading all of it is incredibly frustrating as each part tends to refer to many other concepts that the reader doesn’t understand yet. The word salad of confusing terms and new ideas obfuscates many important core ideas.

Given that preamble, here is the first concept I misunderstood as it flew by my eyes somewhere in the doc: Leave restarting processes to the framework, it is not your responsibility to make it happen, only to configure how that happens. The following is an example of how we learned this the hard way.

One of the services that support our main application is written in Clojure. On a vm or “bare metal” it’s important to use something to restart the application if it crashes. Since Clojure is a JVM language we were able to use the tried and tested “daemonization” concept. We decided to use JSVC to do the actual monitoring/restarting. In practice this means that JSVC starts up 2 processes: one running as root to monitor/restart the application and the other that runs as a lower privilege user and does the actual work of an application. We spent an embarrassing amount of time trying to get this to work in Docker.

When using JSVC in Docker, starting up the container would result in an immediate crash. Where were the logs? Gone. Docker, when started by Nomad (or using docker run --rm which most tutorials recommend) is ephemeral. The instant a container stops, all information not in the image is gone. Therefore we re-invented a technique I’m sure many developers have before: Make the start up script (the “entrypoint” or “cmd”) an infinite loop with a sleep inside. Then use docker exec -it <container> /bin/bash to “sorta like ssh but not really” into the “machine,” run the process manually, and see the errors/logs it produces. Yes, taking out the entrypoint (or cmd), rebuilding the image, and using docker run -it <image> /bin/bash accomplishes the same thing but we didn’t know that at the time.

Unfortunately, using JSVC to start up a JVM app leads to the obscure error “set_caps(CAPS) failed for user …” Doing some research about the error leads to some suggestions of using “privileged mode.” However, everyone warns about using privileged mode. At the time, no amount of searching would reveal the root cause. The short answer was that nobody does daemonization in Docker because it’s forbidden (for good reason) and also not necessary.

Why is it forbidden? JSVC and daemonization are trying to give a user permission to monitor/kill processes owned by other users. This is fine in a virtual machine because there is insulation between the vm processes and the host machine. Having processes kill or manage other processes is fine because they are all in the vm and can’t hurt the host. A container may be thought of as a full other machine but it is not. In many ways it is just another process running on the host with a large dose of Linux awesome to make it run like it’s in another operating system. Having one process be able to kill or manipulate another process (that could be owned by another company when, say, Amazon is hosting a container) is a huge security hole. Therefore Docker correctly shuts down any attempt to do so. Btw, “privileged mode” turns off this protection. 

Why is daemonization not needed? The job of monitoring/restarting/managing is handled by K8s or the Hashi stack or whatever is used to look after the containers. The main (or only) process of a container essentially IS the container. When it stops, the whole thing is restarted per configuration of the container wrangler.

At core here is a Docker philosophy that gets lost amongst all the other documentation: Applications that used to use an entire vm or host are now “just” another process on that host. This allows all the container apps to use resources as needed instead of the wasteful and inflexible assignment of memory and cpu cycles to a vm that doesn’t need them 90% of the time. If a container isn’t using some memory, that memory is completely free to be used by other processes. This is one of the reasons containers are way more efficient than virtual machines or dedicated hardware.

Moving from Asset Pipeline and Sprockets to Webpacker in Rails

Tue, 02 Feb 2021 03:25:00 +0000

In 2018 my Rails 4.x project was given the task of making React “work” inside one of its pages. An additional requirement was to use Webpacker as it had recently been announced as the coming default of Rails 5. We bolted it in to the application but nobody really understood how it worked, interacted with other JavaScript, or how to finish the job of moving to Webpacker.

It’s now 2021, we’ve upgraded Rails a few times, and a ticket to “finish the move to Webpacker” landed on my pair and I. After a lot of searching the internet, copious floundering and cursing, we completed the story in a manner of speaking. It was not easy. Hopefully, this post can help someone save a bit of time.

8 things to know about moving from Asset Pipeline (which uses Sprockets) to Webpacker in Rails:

First, Webpacker is a gem and Webpack is a whole system of bundling assets into modules for “modern JavaScript applications.” Webpacker, the gem, wraps Webpack and provides some methods for using it with Rails.

Second, Modern JavaScript is sometimes called ECMAScript. ECMAScript is the API standard that JavaScript happens to implement. Browsers support ES standards but developers write in a version of JavaScript that implements that version’s ECMAScript protocols. If you’ve stepped away from JavaScript for a bit, this (and everything else about ‘modern’ JS) can be confusing.

Third, Most modern JS development involves Node.js. React relies on Node. Node.js is an event-driven JavaScript runtime. Writing for an event based runtime in the style of a thread based runtime (old-style JS) can be made to work and will be an absolute maintenance, debugging, and upgrade nightmare. A good developer can easily write some very bad code if they don’t understand event-driven architecture patterns. The differences are comparable to the differences between object oriented and functional programming. Spend some time understanding Node.js and the event-driven model before attempting to write anything of consequence.

Fourth, You do not need to move from Asset Pipeline to Webpacker all at once. Nobody sensible moves a legacy Rails application from pre-Node JavaScript and Rails’ Asset Pipeline to modern JS and Webpack in one release because it is not trivial. Any custom pre-Node JavaScript in your application can be converted to Node compatible JS but there are new concepts to grok and gotchas abound.

Fifth, Asset Pipeline and Webpacker can live together as you make the transition. In Rails 4.x, 5.x, and 6.x a legacy Rails application will continue to use Asset Pipeline for the non-Node JavaScript and the assets (images/css) it needs while using Node’s Webpack to handle Node.js assets.

Sixth, You really don’t have to worry about browser compatibility when working with Node.js because of Babel. It translates JavaScript down to whatever version you wish to support. This is nice but means the JavaScript that runs on the browser will be different than the source. Luckily, JS source maps can work with browsers to bridge that bit of indirection.

Seventh, Node.js has serious opinions about scope. This means old-style JS that is used to everything being in the global space will not be able to talk with Node JS unless that code is explicitly exposed globally. Generally, it is considered better to re-write in Node style any old JS that needs to communicate with Node JS code.

Eighth, Ross Kaffenberger’s website is an amazing resource on this subject. He’s compiled his blog posts about Rails, Webpacker, and the Asset Pipeline at https://rossta.net/webpack-on-rails/ This deep dive into all the settings contained in webpacker.yml helped me solve a 2 year old problem in my codebase: https://rossta.net/blog/how-to-use-webpacker-yml.html

To sum up: Moving from old-style JS to Webpack and Node should be an iterative process. Find the seams (see Michael Feathers “Working Effectively with Legacy Code”) and move chunks one at a time to give your QA a chance at catching all the bugs you will create. Many stories make the labor, if not light, then perhaps less crushing.

In Praise of a Big Dumb Import

Wed, 08 Jan 2020 03:08:00 +0000

For 10 years I’ve worked on an application that has, as a major feature, the ability to import a customer’s data from another application we sell. Our customers can click a button to pull over information, review it, and ultimately display it to their customers. Nobody every has to enter information twice, the system is set up with (sometimes custom) rules to determine what data our customer’s customers can see, and everyone is happy.

Well of course not, but it does work rather well for the vast majority of our clients. However, some of our clients are very large and have appropriately large amounts of data they would like to import (sometimes daily). A multi hour import is not great for their business flow.

The ‘problem’ with the import is that it is stupid. It, the import, gets everything from the System Of Record (SOR) every time. Document who’s title has recently changed? Imported. Fund value that hasn’t changed in 15 years? Imported.

At this point it seems like there are some obvious solutions:

All the data is in the SOR’s DB, why not just get it straight from there?
Why pull, why not have the SOR push data when it changes?
Only import the entities that have changed.

If you’ve every worked a place that has 2 applications share a DB, you probably know this is an infamous anti-pattern. Changing and tidying up a DB only used by one application is hard enough but with 2 apps it is crazy easy to break everything by changing a column name or storing data in a different format. The natural outcome is that everyone is scared to change anything about the shared DB so they don’t and duplicate/unused columns and tables abound. Think of it this way: If a SQL database is global mutable state (and it is), then having a whole other application run by a different team that can change that state at anytime is a recipe for disaster.

But that’s okay, we still have 2 perfectly valid solutions: Only import the differences or destroy the import all together and rely on the SOR to push any changes.

The push proposal seems wonderful as the data would flow constantly and no one need ever again wait on a big import. Unfortunately there’s a few problems with that. Our clients are publishing financial data to their customers and they really want to make sure that information is exactly what it should be before releasing it to their clients. One of the nice features of our preview and publish (PAP) app is the ability to see what the website will look like after they push the publish button. Our client can ‘preview’ as any of their customers and check to make sure they are seeing the correct documents and account data. If we adopted a ‘SOR pushes changes to the PAP’ strategy the client would never be quite sure what they are publishing. Sure our client could preview as one of their users and click around but who knows if that data will change right before they press publish? The nice thing about a big dumb import is that it isn’t a moving target like push systems are. We have discussed having the SOR push changes to a ‘holding area’ that could then be imported but then we’re back to an import. It would be a faster import because the holding area could store the data exactly how the PAP system likes it but this plan is a pretty large project and we were looking for a quicker win.

We settled on only importing the differences. Which is, basically, caching and as we all know caching is one of the HARD PROBLEMS IN COMPUTERS. But really, how hard could it be?

Thusly tempting fate, we wrote 450 lines of Clojure that listens to a queue populated by the SOR with creations/updates/deletions, persists those changes, and can answer queries about what objects have changed over a given time period. Now the PAP system, at the beginning of an import, asks the micro service what has changed since the last import and only imports those changes. It is much, much faster and everyone was happy.

Until the bugs came for us. First up was the code inside the SOR exploding b/c the serialization required to push changes out to the micro service would error out when trying to deal with certain objects only used by a small fraction of our customers which is why this bug evaded testing. Then after that was fixed we found, of course, other ‘freak objects’ that didn’t respond well to serialization. That took our change service out of use in production a few times. Each time we’d put the clients back on the ‘dumb’ import they would call in concerned that their import times had gone up and we’d have to explain our issues. Embarrassing and not at all confidence inspiring.

After a few failures in front of our clients the word came down that we need to make these delta imports clever enough to know when the SOR wasn’t outputting proper diffs. We wrote code to automatically switch to a full, dumb, import when we detected a bad state of the caching system. Good idea, right? Except that in a particular case our QA thought they were testing a cached import but they really were testing a full import because some part of the testing environment were messed up and cached imports were magically becoming full imports. That bit of protection ended up hiding a not small bug which escaped into production.

Our next bug had to do with determining which entities could be deleted during a differences only import. The query to determine which objects to delete was complicated when it came to figuring out which linkages between objects to delete. Turns out it had been written in such a way that it unknowingly relied on a bug in removing relationships between entities. When we fixed the relationships bug the query mistakenly destroyed a lot of links between objects. The data was there, but the missing links meant our system (the PAP) thought customers should not see that data and hid it from them. That issue was awful to debug as no one had directly touched the delta handling code in quite some time so it seemed impossible that it had stopped working because of a few releases that only added unrelated features and fixed what we thought were unrelated bugs.

Now anytime we fix the speedy (difference only) import system we have our client care people asking “Well how do we know this time it is really working this time?” They are right to question us. This feature has produced way more bugs than any other feature in recent memory. Usually, after a feature has been out for awhile, you can count on the bugs (if any) diminishing to zero over time. This speed up the import feature, by adding a type of caching, is a permanent ‘sharp corner’ that all developers and testers need to be aware of. It will almost certainly be the direct or indirect cause of other bugs in the future and we’re just going to have to live with that as we have embraced the speed and additional complexity of caching.

Coda: The Big Dumb Import stays in the picture.

We will always, if I have anything to say about it, have the big dumb import because it absolves all sins. It’s a state destroyer: The equivalent of turning it off and on again which, as much we make fun of the practice, is essential when dealing with global mutable state (aka: a SQL DB). Not being the system of record has its perks.

Why Test Your Code?

Thu, 18 Apr 2019 05:38:00 +0000

I was asked today ‘do we need to test this internal page just for developers?’ and that got me thinking about why I encourage testing. I’ve been practicing TDD/BDD since 2004 and this article is the response I should have given.

In order, my top reasons for testing:

Design aid/architecture teacher IF you listen to your tests.
Tests as documentation that executes. They are much more trustworthy than a comment.
Regression prevention. (Prevent bugs in production)

To clarify my first claim that tests teach design and encourage good architecture:

Tests promote the concept of ‘outside in’ development by encouraging simple interfaces as the test is using the object as another object would.
Behavior. Of the objects. BDD. Make sure they play nice:
- Lots of Stubbing/Mocking needed? It better be a controller/commander object.
- Are the Stubs/Mocks exceedingly complicated and difficult to construct? Could be some feature envy going on.
- Plain old objects have few dependencies and therefore don’t need much stubbing/mocking. Since these are fun to write and test, your developers will be encouraged to write more simple objects. Small objects talking to each other. Smalltalk.
Mindful/reflective testing leads to ‘legos not spaghetti’, increased velocity, and higher team morale.

Warning: It took me years to figure out how to achieve balance between value and total test maintenance time. Bad/slow/flaky tests can kill morale.

Some hard fought lessons follow.

Don’t test the view in server-side ‘unit’ tests!

Dom resolution is best handled by Big Browser
The view is not a good or stable object - it talks to everything:
- You’ll have to stub/mock a billion complicated and changing objects so the tests will be crystal in their fragility.
- The view itself changes constantly for non-architecture related reasons. The churn rate can bury a team.
Force all logic out of the view: As little behavior as possible.
Test view helpers.

Absolutely have integration level tests but keep them lean and healthy.

No mocks or stubs
Hit the DB or other persistence
Hit test servers of outside systems
Of course there’s always Selenium…

…Warning: Maintenance costs of Selenium tests are extreme unless you have white-hot automated testers with years of experience learning in the field (hopefully at some other company that has your exact stack). Even with a hardened team of QA/Testers/Breaker/SDETS maintenance of Selenium or any other full path/no stubs test suite will be far more expensive than other levels of testing.

As a way of explaining, here are some questions to ask when an integration test fails:

Is the test server up?
- Did it change recently?
- What did that other team (perhaps in another timezone) do?!?
Has a browser update ruined everything?
Has an OS level update ruined everything?
Is the test DB up?
Was this caused by those intermittent internal connection issues we can’t reproduce and so IT can’t fix?
I’m not sure if this list has an end.

Also, if your project relies on vendor APIs… Oof. Maybe they have a test server (best of luck) or you stub and hope the vendor announces api changes.

Despite all the disadvantages of integration tests, they really do catch a lot of bugs when done well or even decently. Other times they are crazy time sucks with little benefit. It’s shocking how fast one state can transition to another but that is a whole other article.

So, should the developers have tested that ‘internal page just for developers’ given the above thoughts?

Disclaimer: I haven’t done enough React/Angular/New Hotness Javascript to know if any of this advice applies. My experience is mostly with Ruby (sometimes on Rails), Clojure, and Java systems.

Socialistic D&D

Fri, 12 Apr 2019 05:33:43 +0000

Proposal: Socialistic D&D (or D&D more often but costs money yet does many goods for all involved and more)

Cost per weekly session is $10
No refunds for missing a session for any reason
Every week a player can pay or quit with 1 week’s notice.
Quorum is (total number of players divided by 3) rounded to single humans but can not be less than two players and a dungeon master (DM)
Theme of campaign will not be socialism themed - the whole money thing is a scam to get people to show up. The world will still be unjust and perhaps too dangerous for a ragtag band of adventurers.
The DM pays nothing unless they miss a session - in that case they pay $20 and donate their percent to the agreed upon charities.

If a player misses a season their character will be played by an attending member or the DM. Money collected will be used thusly:

30% to the DM because DM-ing done well is a second job.
30% to purchase physical items (maps, figures, models, hand outs) to enhance the game
40% to charities of the group’s choosing

Why?

Like it or not we see a perceived loss of money as terrible thing. 10 bucks a week is 40 bucks a month which is 97 cents short of Netflix and HBO and Amazon Prime combined. Paying that amount will make the player value the game more which is kinda lame but, regrettably, true. To compensate for that bit o’ sadness:

Attendance will improve as money pressure makes players quit or make attendance a priority.
Game play will occur more frequently as sometimes 2 players and a DM will be playing all the characters even if there are 7 players in the group.
Players will be giving 16 bucks a month to charities who need it. Without doing much of anything you are automatically a better person.
As a paid DM they will be under pressure to provide a unique and quality experience.
30% of the funds come back to the group as physical cool stuff or hand outs to skillful and/or lucky players.

The Truth About My Podcast

Tue, 01 Jan 2019 19:03:03 +0000

In 2013 I created a podcast entitled “Are You Famous, Yet?” while drunk with a friend who was ditching engineering to become an actor.

I should start by saying that I’ve always been rather annoyed when people don’t call failures what they are. Calling something a failure doesn’t mean it didn’t have value. Let’s start with this: My podcast “Are You Famous, Yet?” has failed and is ending. There’s a last episode that goes into details and emotions which I will link to when it drops.

Decent podcasts can get between 5-50 thousand downloads per episode in the first 90 days. Some do better and the ones that do worse really don’t talk about it much and that is why I am releasing these cringe-worthy numbers. Our numbers have aways been deeply horrible when compared to the industry standards. The weird thing is: Less than 1% of our guests asked about our download numbers. If they did, I could put them off with some sort of hand waving generalities that were true in as far as anything concrete was mentioned (it was not). One of the reasons we stopped is that continuing felt like lying. Hard working people would re-arrange their schedules to promote shows/performance/plays/etc to our very small audience. Honestly, our audience was mostly listeners the guest brought in (via social medial or other promotions of their own). We spent money spreading the word on Facebook but the $2-10 I could spend on a given episode didn’t ever seem to gain us much. In one of our episodes a social media professional estimated that a small brand should be spending $100 a month to promote itself on social media: Perhaps I could get a side gig to support my podcast?

One last thing before we get to the hard facts, some episodes were downloaded a lot b/c they are old or the great algorithm in the cloud deemed it so. Our most downloaded episode may be our worst: “Interruptions - Episode 3.” Please please do not feel judged if you were a guest on an episode with a smaller number of downloads.

Title	Downloads
Interruptions - Episode 3	455
The Q Brothers’ Burgeoning Empire - Episode 125	423
Heart is Everyone’s Favorite - Episode 9	375
The Good in the Bad - Episode 16	351
Musicals, Death, Sports, and Food - Episode 19	339
Father and Son - Episode 18	335
Dan Taube All Together and Remastered - Episode 12	335
The Almost Tweet - Episode 2	333
Naked and Fearless with Janie Martinez - Episode 17	332
The Aim is Fame - Episode 5	310
That Guy… Who was in that Thing - Episode 4	310
We’re Holding Out for a Hero: Captain Ambivalent - Episode 33	291
New Year’s Eve Reflections and Plans - Episode 8	291
And So It Begins - Episode 1	289
Nilsa’s Story - Episode 7	281
Wisps of Ash - Episode 20	281
Uncle Fun: The Man Behind the Store - Episode 101	269
The Adventurous Life for Everybody - Episode 24	267
Hip Hop Grandma - Episode 21	251
Puccini. Casey Puccini - Episode 22	248
Perpetuum Bruce Lee - Episode 6	248
Andrea Prestinario:The Intersection of Preparation and Opportunity - Episode 11	232
Tales of Awesome Heaviness Part 2: The Relativity of Failure - Episode 15	229
The Let Down - Episode 23	224
Gin Fizz and her Burlesque Commentary - Episode 114	212
The Casual Clawing of Erik Tylkowski - Episode 46	198
Steve Presser: Merchant of Happiness - Episode 66	181
Following Different Avenues:- Episode 13	173
Rachel Finan is Stripper Jesus & Corey Eno Ruffin is Trouble - Episode 170	170
Sad Times w/ Laura and Jake - Episode 59	168
A Home To Fail or Jason Paul Smith’s Unexpected Gift To Artists- Episode 58	167
“Gary” is “Gay” With An “R” In It: The Wisdom of David Boyle - Episode 44	166
Daniel Knox Learned Piano in Public - Episode 190	163
Aurin Squire Just May Be The One - Episode 79	156
The Oral Traditions of Roberta Miles- Episode 77	153
Seth Bisen-Hersh: Theater Raconteur (part 1) - Episode 119	152
The Courtesan and the Cabin Boy - Episode 208	146
RUFY 207 - “Avenge Our Blood” w/ David Finney and Bec Willett	145
The Benefactor’s identical twin! - Episode 55	144
Katherine Biskupic and Leah Frires are “Pure and Weary” - Episode 112	141
Douglas Post is All Aspects of Theater - Episode 113	139
Freckles and Camp Grounded - Episode 175	138
Joe Foust and Laura Love Door County - Episode 135	138
Christine Lin Has Engineered A Career in Performance- Episode 56	137
Changes with Jake and Laura - Episode 206	137
Jeremy Wechsler’s Outward Going Charisma (Part 1 of 2) - Episode 96	137
John Weagly: Dr. Goat: Goat Doctor - Episode 116	135
Tana Karo and the Acrobatica Infiniti - Episode 128	133
Dana Lynn Formby and her American Beauty Shop - Episode 131	131
Happy Irish Cows Make The Best Damn Butter with Amanda and Tim Winters- Episode 34	130
Psychologist, Playwright, and Troublemaker: Paul Pasulka - Episode 129	129
Ken Davenport Produces Art into Existence - Episode 123	127
Rori Nogee Will Get Back to Broadway - Episode 121	126
Farah Williams Loves All of Chicago - Episode 199	126
Thom Britton Imagines Sideshow Spalding Grey - Episode 126	125
Mosque Alert and More with Fouad Teymour - Episode 124	125
Hugs During Wartime - Episode 61	125
Tim Mooney is Moliere Than Thou - Episode 82	124
The Magical Realism of Emmi Hilger - Episode 50	121
Going Balls To The Wall With Valerie Hager - Episode 54	121
Bryn Packard, Nilsa Reyna, and 360 Video - Episode 138	117
Tim Barnes Makes With the Smart Funny - Episode 172	117
Tony Mendoza Has Wanted to Make Movies Since “La Bamba” - Episode 204	116
John Abbott, Buddy King, Holly McGuire & Connie Foster - Episode 186	116
John Battles: “Let’s All Go To Uncle Fun” - Episode 196	116
David “The Rock” Nelson - Episode 37	115
Barbara Lhota and the Intergalactic Dog - Episode 130	115
Melanie Adcock, Bryan Sperry, and the “House of the Apocalypse” - Episode 188	115
Josh and Caroline Beadle and 3 Brothers Theater - Episode 163	114
Maria Vorhis, Sexpert Franzen, Clowns, etc. - Episode 179	114
Amy Sumpter: Comic and Beastie Boys Tribute - Episode 115	113
Marc Frigo is a Platinum Mixer - Episode 137	112
Ron Keaton Made His Own Churchill - Episode 117	112
David Fink: OUTspoken - Episode 145	112
Rev. David Holcombe Returns with More Films - Episode 162	112
Madelyn Sergel Knows the Many Sides of Production - Episode 160	112
Jason Paul Smith In A Beulah Box - Episode 100	112
Klea Blackhurst on Yodeling Muchachas, Ethel Merman, and playing Shelby Cross - Episode 27	111
The Cast of Three Cat’s Holiday Radio Show - Episode 209	111
Things Get Wild with Storyteller Erin Diamond - Episode 65	110
Amy Crider is About to Re-read All of Proust - Episode 205	110
Seth Bisen-Hersh Part The Two - Episode 120	109
Bryn Packard and Nilsa Reyna Unionize - Episode 139	109
The Dirty Details of Producing “Punk Grandpa” - Episode 118	109
Playtime - Episode 10	108
Jerrod Bogard and the Reality of Writing - Episode 49	108
Laura does Postmates, Baltimore, and Door County - Episode 185	108
Jacqui R√™go and the cast of Baby Hubris - Episode 182	108
EntrePoNegroes, Blaculas And The Mind of Reginald Edmund - Episode 92	108
The Lost Bar Mitzvah Speech of Blake Levinson - Episode 68	108
Michael Jones: Young American does Good - Episode 127	107
Owen Hickle-Edwards and Andrew Root from Runcible Theatre - Episode 181	107
Amy Crider Can Not Die on Saturday Morning TV - Episode 70	106
Barbara J. Wells: Playwright, librettist, lyricist, screenwriter, novelist, producer - Episode 169	106
Bernardo Cubria’s Tales Of The Tex-Mex Mafia- Episode 52	106
Tales of Awesome Heaviness Part 1- Episode 14	164
Caryl Pohlmann is the ‘Punk’ Grandma - Episode 168	106
Elise Mayfield is a Master Chef and Actor - Episode 72	105
Laura Interviews Jake! Also: 2016 Art Retrospective - Episode 165	104
Jim Lasko and the Redmoon - Episode 143	104
Episode 36	103
Jonathan Pitts, 2 Festivals, and a Chimpanzee - Episode 150	103
I Do Everything Better Naked: A Talk With Kevin Urban - Episode 26	103
A Shapely And Beautiful Hostful - Episode 88	103
The Alien From The Planet Fun - Episode 84	103
Schoen Smith Does Theater/Film/Directing in New York/Chicago/L.A. - Episode 192	102
Ken Coughlin: Musician, Actor, Punk Grandpa - Episode 122	101
CJ3 Likes a Nice Catsy - Episode 76	101
Ele Matelan, Josh Zagoren, Foley, Horror, and Wildclaw - Episode 174	101
Sara Maslanka: Artistic Director of Chicago Danztheatre - Episode 183	101
Richard Henzel plays Ronald Reagan and Mandolin - Episode 166	101
The William Oliver Watkins Experience - Episode 25	101
Josh Wolff makes movies and Bryn makes plans - Episode 41	100
The Honey Brown Goddess - Episode 63	100
John Ritter: Sex Object w/ Eileen Tull - Episode 85	100
Shariba Rivers Has A Lot Going On - Episode 193	100
Mariah Karson’s Powerful Dad Joke - Episode 176	100
Talking Taboo, Los Improviachis, and Making “Shift” Up w/ Nelson Velazquez - Episode 178	100
Dolores Diaz, Wendy Schmidt, Lisa Scott, and Laura Scruggs Write a Holiday Show - Episode 210	100
Zoe Sjogerman, King Lear, and the Eleusis Collective - Episode 136	100
The Ms. Subways Mystery - Episode 86	99
Brittany Meyer: Fake TED Talks, Arm Wrestling, and Strip Joker - Episode 203	99
Bryn and Nilsa Leave the Show - Episode 111	98
Corky Siegel Plays Chamber and Symphonic Blues - Episode 159	98
The Parallel Universe Door-Slam Farce and Other Worlds of Leonard Franks - Episode 30	98
Dan Rybicky and 10 Years of Almost There - Episode 161	97
Don Hall Directs All the Events - Episode 158	97
Creative Jakub - Episode 87	97
Ironically Mortified with Jill Howe - Episode 74	97
Captain Ambivalent’s Ongoing Adventure - Episode 148	97
The Man Who Would Be Josh Force: Justin Shaffer - Episode 99	96
The Once and Future Performink w/ Abigail Trabue and Jason Epperson - Episode 144	96
Mariana Green Directed Davy Tomei as Punk Grandpa - Episode 98	95
Bryn, Nilsa, Laura, and Jake - Episode 157	95
Rachel Edwards Harvith - Episode 110	95
Late Nite Slam Futurist: Lisa Buscani - Episode 151	95
Bryn and Nilsa 200th Episode Spectacular!	94
Bureaucracy and Yoga - Episode 104	94
Meghan Beals. Artistic Director of Chicago Dramatists - Episode 109	93
Mitch O’Connell Draws Everything Sexy - Episode 167	93
Ada Cheng’s Change - Episode 198	93
Ronen Kohn and Time Crash - Episode 149	93
Greg Allen is in his Third Decade of Neo Futurism - Episode 141	93
Carolyn Hoerdemann was/is a Time Traveling Rockstar - Episode 133	92
Chris Jones: Chief Theater Critic of the Tribune - Episode 154	92
Black Bryn? White Warren? Warren Phynix Johnson - Episode 47	92
Jeremy Wechsler Is A Theater Revolutionary (Part 2 of 2) - Episode 97	91
Patricia and Kendra and Leela - Episode 90	91
Sean Masterson and the Theater of Magic - Episode 132	91
New Place and Stuff in Our Lives - Episode 197	91
Jamey and Susan Barnard - Episode 195	91
Anna Lucero & Mary Rose O’Connor: A Little Nod and a Big Middle Finger to Beckett - Episode 142	91
Melissa DuPrey and a New Intern - Episode 105	90
What has Laura Force Scruggs been doing this Year? - Episode 156	90
Man Interviews Cat - Episode 78	90
John Frigo and Sherri McGlothlin’s Artistic Life - Episode 40	90
Movement and Viewpoints and Silliness and Slugs: A Mildly Explicit Hostful- Episode 60	90
Lori Myers and Not In Our House - Episode 146	90
Rhonda Findley and David Gordin and Fun Rock’n and Pop City - Episode 164	89
David Zak, The Bailiwick, South Korea, Hammond, and Priscilla - Episode 171	89
Bob D’Haene Hasn’t Been To Bed, Yet- Episode 64	89
Andrew Seller: Laura’s Muse - Episode 89	89
Meredith Lyons Will Kick You In The Throat - Episode 28	89
Cheryl Coons: Story Catcher, Guild Representative, and Resident Playwright - Episode 134	88
Les Scruggs at Snake the Lake - Episode 191	88
“The Finder of the Lucky Devil” and Megan Mackie - Episode 187	88
Corson Barnard: Straight out of College - Episode 212	88
One Year Anniversary Hostfull - Episode 51	88
Aamira Martinez has not seen Blade Runner - Episode 35	88
Jay Nagle’s Ecologic and Epic Life - Episode 103	87
On the Importance of Fun - Episode 80	87
Santa is the Baum w/ Laura and Jake - Episode 202	87
Laura, Jake, 2017, and Beyond - Episode 211	86
The Heart of it All with David Holcombe - Episode 43	86
A Structured Conversation with Elizabeth Cambridge - Episode 194	86
Jacob Reyes Brings His Own Theater - Episode 201	85
An Oddly Down Bryn - Episode 57	85
Mrs. Scruggs’ Wild Ride and Ms. Reyna’s Magical Times - Episode 42	85
Leslie Scruggs and the wily accountant - Episode 39	85
It’s Jake’s Birthday - Episode 91	84
Tanner Melvin and the Elgin Fringe - Episode 152	84
New Beginnings - Episode 102	84
Bookings! Mischief! Distractions! - Episode 38	84
Leigh Barrett has Blade will Babe - Episode 140	84
July 13th, 2018 “Cabaret on the Lake” - Episode 227	84
I’m Hotter Than You Are - Episode 67	83
We found Lindsay Porter’s Lost Internet Presence - Episode 93	83
3 Texans Take New York w/ David Davila/Benjamin B. Wright - Episode 45	83
“Uncle Fun: You’re The One” Update - Episode 177	83
Dax Valdes Knows How To Hustle- Episode 62	83
Danger Circus at the Last Abbie Fest - Episode 147	83
Working Yes, Cash No - Episode 106	83
Megan Mackie and The Princess Peach Conspiracy - Episode 173	82
Short Update on our Last 5 or 6 Things - Episode 226	82
Felisha McNeal Chooses Dream Explosion - Episode 108	82
The Condition of Femme - Episode 217	82
Tonya, Nancy, Priscilla, and Jon Martinez - Episode 153	81
Erika Arneson Puts The Art In Barter - Episode 48	81
Mare Swallow’s Life Adaptation - Episode 155	80
The Devil and Anna Lee Anderson - Episode 83	80
Rick Olson, Stephanie Palko, and The Edge of Orion Theatre - Episode 189	80
Come To Play and Make Something Cool - Episode 81	78
Maureen Smith and Daniel Johanson of Scapi Magazine - Episode 228	78
Jake, Laura, Ladie M, Uncle Fun, and Stand Up Comedy - Episode 215	78
A Modicum of Modesty - Episode 94	77
Tim Ferro: Best Color Man in the Business - Episode 53	77
Baltimore Here We Come - Episode 229	77
A Lifetime of Music with Jon Spiegel and Some Storytelling - Episode 218	76
“Wicked, Immoral, Utterly Bad!” with Pete Blatchford - Episode 230	76
‘Do Not Submit’ to Karen Genelly - Episode 184	76
Ellyzabeth Adler and Danztheatre - Episode 180	76
Mistakes Were Made and Triumphs Were Wrought- Episode 75	75
Taking Care of Each Other and Ourselves - Episode 69	75
Finding Your Happiness With Geoff Barnes - Episode 32	74
Kelly Bolton as ‘Andy Drodgenous’ - Episode 107	74
The Price of Fame - Episode 95	74
Rosina and Ryan Snow, Jazz, Song, and the Law - Episode 216	74
Chris Kriz Designs Many Sounds for Many Theaters - Episode 31	73
Big Scary Fur Teeth, Love Rat and Vegetable Candy- Episode 73	72
The Flattening of the Pyramid- Episode 71	70
Amy Crider, Self Producing, Leda, and Metadata - Episode 213	69
Uncle Fun: You’re the One premiere Q&A&Performances - Episode 225	69
Drugs, Richard J. Miller, The Facts, and Society - Episode 214	67
June 8th Three Cat Cabaret on the Lake - Episode 224	67
A Hostful Episode - Episode 29	66
Thom Britton and Jonathan Pritchard Play with the Dead - Episode 231	65
Chels Harvey talks social media, podcasting, and writing - Episode 220	61
Uncle Fun Documentary is Done! - Episode 219	60
Daniela Thome’s Journey to “Not For Sale” - episode 233	60
Surreal Cupcake Comedian Martin Felshman - Episode 223	57
Lauren Sowa of Form & Pressure Films - Episode 235	55
Fairy 101 at the WasteShed w/ Laura Scruggs - Episode 222	54
David A. Holcombe and “Blacksite: The Musical!” - Episode 232	54
Laura’s New Documentary Subject - Episode 221	49
Cabaret on the Lake September 14th, 2018 - Episode 234	46

Notes:

These numbers are for all time, not just the first 90 days after release.
I realize that a non-insignificant number of these downloads could be ‘bots.’ Sigh.
Episodes that were re-released had their numbers combined.
Libsyn, my podcast host, made getting these numbers difficult so I wrote a Ruby program to merge the files they provided and it was way harder than I thought for various super nerdy reasons. That tale could be a whole post…
The biggest problem of the podcast is that no one wants to spend an hour every week with an artist that they have never heard of and sometimes have no ability to experience (Ex: Chicago only performances/performers). I genuinely liked the artists and, like my father, most had never had someone study their career and ask thoughtful questions in a long form interview. Getting out the story of the vast underclass of artists (underpaid not under-talented) felt like a worthwhile goal but getting people to listen was a problem. We were not “sticky.” Listeners would tune in for an episode about their cousin/friend/workmate/boyfriend who was in theater/a band/to eating fire and then disappear.

The Tears of an Apprentice

Thu, 25 Oct 2018 21:00:01 +0000

To sum up my non-standard path to development could take some time but the short version is H.S. Physics teacher => 3 months as an apprentice => Thoughtworks => Obtiva => Backstop Solutions. Because of this, when I was just starting out at ThoughtWorks, I was actually an apprentice but paid and labeled as a full developer. During that time I was consulting at a fortune 500 company with a team of employees and ThoughtWorkers and employees. One particularly rough day I was admitting ignorance again and again and looking very stupid because of it. One of the key patterns of being an apprentice is “expose your ignorance” and yet it is excruciating. As adults we are used to being good at things and to go back to being embarrassingly bad at something is such a mental strain that most people don’t do it. Try learning a second language or a first musical instrument and you will feel this pain.

This may have been the day that I told my pair (who worked for the client and could get me fired) that I though a ‘<td>’ in some HTML made a call to the database. In my defense it is a table data element so it wasn’t a terrible guess…

Many more mistakes like that were made and then it was time for lunch. I was feeling pretty bad but we all had a nice lunch on uncle ThoughtWorks and all was, if not well, then better. At the end of the lunch we played “rock, paper, scissors” to determine who expensed the lunch. I’m not sure if you’ve played this game with people who play it all the time but they get fast and oddly great at a supposedly random game. And so they crushed me so bad that someone had to explain that I had lost (it all went so fast). On the way back to the office I was in a state of shock: I had gotten used to being dumb at programming but getting smacked down at lunch wasn’t even on my radar. I began to fear that leaving my tenured teaching job may have been a terrible mistake. As we drove on through the oversized streets of the job’s medium sized midwestern town I had to look out the window and pretend to be interested in the view so they wouldn’t see the tears in my eyes.

This is an event I think about a lot when training apprentices and junior developers. I try to say often: “You will feel overwhelmed and terrible and that’s what everyone feels and it’s fine. If you were doing a bad job, relative to your experience, I would tell you.”

Moving to Github Pages with Jekyll and fixing a pagination bug

Sun, 09 Sep 2018 21:38:01 +0000

In an effort to make my online presence less dated and more organized I’ve made jakescruggs.com a real site and not just a forward to a Blogger instance. GitHub pages was so dead simple and free-ish (if you want the source control of your site private then you need to pay for a private plan) that it would be hard not to use it. I’ve been interested in the generating html static sites movement for awhile now and here’s how I saw the choices:

Take Jekyll and the limited plugins GitHub Pages supports and get deployment every time I push to GitHub with no servers to maintain.
Roll my own hosting solution (AWS) with any damn static generator I like
Netlify which has a pretty nice free plan www.netlify.com/pricing

Ultimately the fact that I already had a GitHub private plan, am a longtime Ruby programmer, and I figured the cost of change could safely be deferred until a latter date. I went with GitHub Sites and Jekyll (which is written in Ruby). The other static site generators features and speed didn’t seem compelling.

Generally I’ve been delighted with the ease of conversation from Blogger to GitHub/Jekyll. Blogger has a nice feature whereby you can download all of your text-based content as one xml document. I found a bunch of 6-8 year old Blogger xml to markdown projects that were a bit out of date but this Node based one did a very good job: github.com/palaniraja/blog2md

After that it was just figuring out some minor syntax things and I was pretty much done except for pagination. The state of the GitHub official supported pagination plugin is a bit weird right now. The newer re-write of the standard is not supported by Github pages. Documentation for the old paginate plugin is spotty and slightly different than the newer one. I found that creating a pagination list tricky given the instructions at https://jekyllrb.com/docs/pagination/ which suggested this pagination link producing snippet:

  
    {% if paginator.total_pages > 1 %}
    <div class="pagination">
      {% if paginator.previous_page %}
        <a href="{{ paginator.previous_page_path | relative_url }}">
          &laquo; Prev
        </a>
      {% else %}
        <span>&laquo; Prev</span>
      {% endif %}

      {% for page in (1..paginator.total_pages) %}
        {% if page == paginator.page %}
          <em>{{ page }}</em>
        {% elsif page == 1 %}
          <a href="{{ paginator.previous_page_path | relative_url }}">
            {{ page }}
          </a>
        {% else %}
          <a href="{{ site.paginate_path | relative_url | replace: ':num', page }}">
            {{ page }}
          </a>
        {% endif %}
      {% endfor %}

      {% if paginator.next_page %}
        <a href="{{ paginator.next_page_path | relative_url }}">
          Next &raquo;
        </a>
      {% else %}
        <span>Next &raquo;</span>
      {% endif %}
    </div>
    {% endif %}

Unfortunately the first page link is wrong when on any page above 2 so I went with:

  
    {% elsif page == 1 %}
      <a href="{{ site.paginate_path | relative_url | replace: '/page:num', ''  }}">
        {{ page }}
      </a>
    {% else %}

Not too terrible to fix. Like I said: The Jekyll/GitHub Sites experience has generally been a delight.

Getting my url to be jakescruggs.com instead of jscruggs.github.io was a snap. I followed the directions to configure Hover and GitHub created a CNAME file for me in the project. I was done in minutes.

I’m currently considering if I want to have comments and if so I’m leaning towards “Staticman.” Staticman is very interesting. It uses javascript to fire comments at a public Staticman API (or a private if you prefer to host your own) that then pushes those comments into your GitHub repo and that, of course, triggers a publish of your site. GitHub as DB.

The 60 inch Pair Station

Thu, 16 Nov 2017 00:27:40 +0000

My company was thinking of moving out of the Willis Tower until they made us an offer we didn’t refuse: A large amount of money to remodel and a bigger space. One of the outcomes of this ‘free money’ was a bunch of 60 inch flat screens occupying the walls with no real plan for usage. Pretty soon postcards and calendars started appearing on their surface.

Which seemed a bit of a waste.

One day Toby and I were pairing and happened to be about 10 feet from one of the unused TVs so we pulled our desk up close to the wall and used the 60 inch for a display. It was wonderful. We could spread out all our windows so much so that we didn’t even need workspaces. When it came time for help and/or a code review the whole team could easily read the code. Soon we got a rollable stand for the monitor and were able to use it at a more appropriate height and anywhere we wanted.

We loved the collaboration inspired by the first 60’’ monitor so much that we rescued another 60 to be a part of a second pairing station. We could have 4 developers sitting side by side helping each other while communicating effortlessly. Collective code ownership felt way more real when all the code was huge and readable from a distance. When we had a sustained effort to build a micro service it was delightful to have the pair working on the API sitting right next to the pair consuming the API. At any point 1 of 4 developers could glance over at the other screen to see either what they expected or question something they did not understand. Through this interaction many bugs were caught long before they were committed to the codebase.

Also, it’s fun to watch the latest internet videos on a huge screen for team bonding. You do need to be mindful of disrupting the other islands/teams.

TVs today have gotten so inexpensive that this really doesn’t cost much. 150 bucks can buy a rollable stand for a large monitor and the monitor itself can be less than a grand. A company could make a statement about how much it values pair programing by spending more on the shared monitors than on individual ones. 4k HDR would make for a pretty sweet pairing station…

Some developers don’t like working on a 60 inch monitor because they feel exposed as anyone can walk up and criticize your code. That can be pretty intimidating to a seasoned developer, and junior developers may feel downright scared. Most times this is just a perception and everyone behaves like adults and collaboration is aided. If this is an actual problem then using big monitors will expose toxic developers and they should be corrected of their unintentionally harmful comments or fired. Protecting your people is just as much about finding the bad as it is about promoting the good.

I’ve been working every day with this huge monitor and I love it. I love that it encourages the communal nature of code. We all own it. We are all in this together. The message is unequivocal: The team will solve the problem as a group.

Clojure ‘def’, ‘declare’, and ‘concat’

Mon, 01 Feb 2016 05:02:00 +0000

I hit the Aeron hard and asked my pair Liz “what’s up?” I’d been sick and she had been working on a tricky Datomic datalog query in my absence. She claimed to have solved the whole problem but the tests would not pass when run as a whole vs. when run individually.

A classic problem with a big Clojure twist. I, being no stranger to this terribleness, eventually shook the sick out of my head and asked if she had tried running the singular test file by itself. Good news: it failed too. What? I’ll explain. We use a classic trick here at Backstop when running tests in the Clojure REPL: ‘:reload-all’

(require '[clojure.test :refer [run-tests]])
(require 'your-ns.example-test)
(run-tests ‘your-ns.example-test) ; First time

(require 'your-ns.example-test :reload-all) ; Picks up new changes in the files
(run-tests ‘your-ns.example-test)

Otherwise we’d spend all day waiting for the JVM to start up every time we run our tests. It’s a cool trick but it has its limitations. We were clearly dealing with one of those limitations as the tests would fail once in a fresh REPL but succeed on a second run. Those who’ve already guessed the answer please apply for a job here. For all us other mere mortals who need to think it through I’ll present this example of what we were trying to do and what happened:

(declare a-vector)
#'user/a-vector
(def time-bomb (concat [1 2 3] a-vector))
#'user/time-bomb
(def a-vector [4])
#'user/a-vector
time-bomb
IllegalArgumentException Don't know how to create ISeq from: 

clojure.lang.Var$Unbound  clojure.lang.RT.seqFrom (RT.java:505)

But if you asked for time-bomb again:

time-bomb
(1 2 3)

Everything is, er, fine?!? Except ‘a-vector’ has disappeared and a ‘function’ has returned different things for the same lack of input. So, of course, “concat” is part of the problem here. As Stuart Sierrapointed out in http://stuartsierra.com/2015/04/26/clojure-donts-concat, “concat” is a lazily evaluated join so it can hide many bombs.

If we had instead used “into,” which is not lazily evaluated, all would have blown up in a nice normal manner:

(declare b-vector)
#'user/b-vector
(def now-bomb (into \[1 2 3\] b-vector))
CompilerException java.lang.IllegalArgumentException: Don't know how to create 

 ISeq from: clojure.lang.Var$Unbound,

 compiling:(form-init1552771854893412833.clj:1:15)

Well, not a great error but at least the line number would have given us a clue as to where the problem was instead of the stack trace starting where ever the “time-bomb” was evaluated.

However, “a-vector” is defined by the time “time-bomb” is evaluated. So why doesn’t the lazy “concat” use the available “a-vector”? “Concat” seems to freeze the temporary nature of clojure.lang.Var$Unbound and not evaluate a later defined “a-vector.” Weird. Is this a bug? Or just a known evaluation order thing? I verified this behavior in 1.5,1.6,1.7, and 1.8 so it seems here to stay.

In case you are wondering, without a “declare” this code would blow up right away on the right line number:

(def nope (concat [1 2 3] nothing-at-all))
CompilerException java.lang.RuntimeException: 

  Unable to resolve symbol: nothing-at-all in this context, 

  compiling:(/tmp/form-init1552771854893412833.clj:1:11)

Why did we use declare instead of just order the code so it compiles? As a team, we’ve adopted a code standard where we try to push private functions down to the bottom of a file and “declare” helps us declare things that will be coming to the compiler/jvm/macro-magic-factory. It works really well with “defn” so we applied similar principles to public “def”s (for use outside the namespace) and “def”s we didn’t think should be part of a namespace’s API.

The combination of testing in the REPL, code conventions, the lazy nature of “concat,” and when “def” gets evaluated created a lot of confusion for us. Hopefully, this blog post helps you figure out some similar problems.

Lutz’s Super Truthful Murder-ball adventure

Wed, 25 Nov 2015 21:06:00 +0000

For backstory checkout out the previous DnD post.

Landing in a pile on a large stone table in a massive room from out of a swilling green magic portal is very disorienting. However, it did lead to Lutz shaking off his magical blindness. First sight: Many intense dudes some covered in lacquer armor — Lutz didn’t even know lacquer was a thing you could make armor from. While Lutz pondered the truth of his eyesight, Lord Valparaiso threw a little fit. One: He was mighty pissed that his evil portal trap has ensnared him in addition to our heroes. Two: The portal had clearly been hijacked to another mysterious and far away land. The land was so strange and far from their home that Raest couldn’t feel his god Palore anymore which meant no spells. Essentially this changed his role in the party from from healer/crappy fighter to, um, crappy fighter. That single set of footprints on the beach are not from Palore carrying Raest during his darkest hour.

The team had many questions but lacking a common language they were ignored, shoved out of the big room, down a hallway, and into a smaller room with a chalkboard. A very painful session of explanation ensued. Drawings of stick figures and angry gestures soon left both sides exhausted. The time had come, once again, for Lutz to put the team on his back. Working with noted jerky-owner and consumer Bran they managed to extract salt from beef jerky infused water with heat from a torch. That pinch of salt combined with a pinch of soot plus Lutz’s NEW FOUND ARCANE TRICKSTER POWERS meant he could cast a Comprehend Languages spell. Clutch.

Now that they could understand what was being said, via Lutz, the strangers in a strange land understood that they must play a spirited game of foots-ball with big bats and a medicine ball. There would also be giant walking trees that shoot hurt-y needles if you get too close. Hitting other players was allowed and encouraged but the ultimate goal was to get the 2 foot wide medicine ball into the goal by any means necessary. Winner gets not to die. Best 2 out of 3. Classy.

Our ambassador/coach/warden gave the team a (perhaps) last meal. It kinda seemed she might have an interest in seeing team Lutz win. Was betting involved? Who’s to say? Mind your own business. The important part of this last feast was that everyone got a rest and Lutz hid away a small amount of alcohol.

Upon reaching the field for the first match the team saw their competitors. From far away they appeared to be a bunch of imps and a man. As they came closer the dread increased as our brave band of adventurers saw one 9 foot tall center and a bunch of good sized dudes.

At the start of the match more bad news came pretty fast: Clustering around the ball made the team susceptible to a sleep spell from the safely 50 foot back magic user. The other team had “forwards” running down the wings, defenders hanging back, and an actual strategy. Peren, Lutz, Bran, and Raytheon had more of a “swarming bees” strategy that kinda played right into team bad guys’ “cast a sleep spell and send a man down the side lines” strategy.

You might think the gang got out of this one with some clever thinking, but no, they mostly just whacked everything nearby which included the ball. Ray-ray, in particular, took out his frustrations over losing all of his promising new cult by smashing every opponent excessively: Often from behind. Somehow this aggressive flailing got the ball past the team jerk face’s front line and rolling. The big bad, being not very big on smarts, crushed the ball towards the wall which was about 90 degrees off from his target goal. Lutz used his superior speed and agility to do 2 things in the time another would merely perform one. He wake up the sleeping members of his team and flanked the opposition. Bran got all “angry barbarian” and made like Lutz with the 2 to 1 actions. Unfortunately the other team had a double attacker too - lots of hustle displayed on the pitch that day. The first match was won by Lutz picking up the ball, using his crazy moves to break the ankles of the other team, and bursting on through to the goal.

During the break Lutz cast a False Life upon himself with his fly arcane trickster powers using the dram of alcohol he had acquired from the “last supper.” Palore, wherever he may be, did not favor Raest with the heal spell he so desperately needed. Only a jerk would have pointed out that necromancy had succeeded where religion had failed. A bit more rested than before the team braced for the coming match. How would the other team solve the “Lutz problem?”

Well how could you? He’s just too damn fast and agile. Like they used to say about famed foots-ball player Mycroft Jordan-Hammer: You can’t stop a Lutz, you can only hope to contain him. Once again Lutz woke up sleepy members of the team, picked up the ball, and made a break for it. Problem was that this time a terror tree decided to get between Lutz and the goal. Now a normal thief, pick pocket, 2nd story man, or even assassin would perhaps have turned tail at that point. Lutz sucked up his courage, cast “Thunder Wave,” and knocked the terror tree aside. An entire walking tree pushed back 10 feet by a skinny studded leather wearing punk of an 18 year old. Just picture it. Now picture a bunch of jerks chasing Lutz for a few long seconds as it is became apparent to all that it was going to be a 2-0 sweep. Clutch.

Now that they won the right not be slaughtered for a god’s favor, how will these heroes be treated in a land even Palore has forgotten? Tune in next time to find out.

Our Purely Functional Microservice in Production

Tue, 17 Nov 2015 23:44:00 +0000

Recently I was interviewing a candidate for a job on my team at BackstopSolutions and he said something like: “I read your blog post about Microservices and was wondering if the lack of follow up meant the project was a failure.” He actually expressed this sentiment in a much nicer way as this was an interview but still it got me thinking: ”I should do an update on the status of our microservice and explain how it could be purely functional.”

The service is implemented in Clojure, uses Datomic for persistence, and integrates via REST.

The service has been in production for several months now and has gone through a number of versions of its API. Implementing HATEAOS worked out well as it allows us decouple having to keep the service and front end in lock step. Since this is a microservice, we could keep both applications in step with each other as we own both ends of the interface. However this co-ordination is not without cost. Release dates slip b/c of bugs in either system. There are less bottlenecks in production releases as the front and back end can be deployed to production independently of each other. Not having to co-ordinate releases is a wondrous decoupling. It didn’t take long to implement HATEAOS and maintenance costs for the discoverability aspect have not materialized.

Datomic is a special snowflake of a database and that it why I believe it is the future. That is a bold statement deserving of a whole other blog post so I will merely list a few of Datomic’s features that we exploit:

It never forgets. Think of it like source control but with transactions replacing commits.
Time is a first class citizen. This is really an outgrowth of the above feature but is worth pointing out on its own.
It uses the incredibly compose-able Datalog query language instead of SQL and can include clojure.core expressions in its queries.
It can easily be used as a graph database.

Those are some crazy advantages. Since Datomic never forgets, ANY query to our service that specifies a time can never have a different return value. That is what I mean when I say that our service is “purely functional.” This enables the Rails app that works with our micro service to cache every query it makes FOREVER. The service without caching is still crazy fast considering the incredibly complex queries is has to run. Like any graph database, speed is determined by what percentage of the graph a node can access. Even if a customer stretches the engine to its maximum, our ability to cache forever means they will take that hit but once. Of course you can query our microservice with no time specified where it will default to now and not cache. This is used only for debugging and is never used by our Rails application.

The graph database aspect of Datomic combined with its understanding of clojure.core enable us to spend days instead of months (or even years) implementing custom business logic. Backstop Solutions is a Business to Business (B2B) Software As A Service (SAAS) firm. In B2B there is but one constant: All business’ processes are unique. Period: End of story.

There may be some similarities and generalities between companies in the same competitive space but these similarities are revealed to be mostly illusionary when you try to implement specific customers. Drill down into any business’ rules and you will find unique requests from every client. Think of it this way: In an efficient market what use is there for complete duplication? The clients who’s processes are a competitive advantage hate any SAAS offering that can not implement aspects of their system in the SAAS product. With this microservice we can. The actual challenge is to get clients to simplify their rules enough so that they themselves can understand the rules. This may seem confusing until you consider that Backstop Solutions has clients with many offices in many different lands so we are often the first people to implement a logically consistent system that fits all of their needs, people, and processes.

Datalog is a subset of Prolog. Prolog is a logic based language as is SQL. Where it crushes SQL like a bug is its compose-ability. You can create a bunch of “rules” and mix and match these rules just like you are used to composing functions. This is amazing. I can not stress this enough. I continually implement stories in a fraction of the estimated time because I discover that most of the query is already written and I just have to tack a little on the end, or beginning, or middle.

What is even more wonderful is that even if I ever get into a tight corner with Datalog I can fall back on all the functions in clojure.core. How many times has it infuriated you that the version of SQL you are using can’t support a completely logical function and so you have to do something horrible? Now imagine being able to use all of clojure.core in SQL. Exactly.

I should stop and point out that I had the idea for this service 4 years ago during a conversation with David Chelimsky. As he described a problem he was solving with a graph database I realized that we had a completely different but functionally similar problem where I work. Thanks for always being inspiring David.

Lutz’s completely accurate DnD summary for those who weren’t there

Tue, 10 Nov 2015 03:14:00 +0000

The “Curious Case of the Tiny Thieves” is being joined in progress and so a little backstory is necessary:

The dexterous and rather handsome rouge Lutz was investigating these “Tiny Thieves” with his band of mostly merry adventurers: Raest a cleric of Palore, Bran the mighty barbarian, Torment the super naive atheist paladin, a wild magic user called Very Forgettable Name, and Iesha or Aida or Ada AKA: the old lady magic user who casts a lot of sleep spells. They discovered the little thieves trying to abscond with their solid gold cat statue which they received as a gift for returning a talking cat to an old lady who was really a dragon in human form — long story.

Turns out the whole town hated these mini scofflaws as they has been on a veritable spree of the crime variety. A reward was offered and pursued by our fairly fearless heroes. As far as they knew the criminals were: 6-8 inches tall, never seen in full light, and immune to sleep spells. Lutz made a super cool trap for these minute miscreants but they avoided it perhaps because the gang had left the lights on. Seems obvious in retrospect… They did find tiny foot prints in the fireplace the next day! They were also found the same footprints the next room over where a very expensive necklace had been stolen. Tiny thieves: 1. Adventurers: 0.

That was all prologue, believe it or not.

Upon investigating the roof and dirt surrounding the “Sleepy Dragon” the team supreme found evidence of rope used to climb down the chimney and footprints leading away from the inn. Following the trail of regular sized footprints the gang pretty much got lost right away. Sigh. The trail was definitely headed south and that’s about all they knew. In that direction lay: The bad part of town, The duke who had suspiciously never been robbed, and a new unknown church that ministered to the poor in said bad party of town. “Why not stop in at the working class church and ask around?” thought pretty much everyone. Raest, who apparently has more money than sense, decided that 5 gold was an appropriate bribe/offering to a dirt poor chump. Not 5 copper, mind you, but 5 gold. As in 500 copper. Bran had been invited in to the small place of worship by Raest in a move that was so very sensible it belied his previous insane tipping/bribing strategy. I feel like I’m focusing on this too much… But seriously 5 gold. FIVE DAMN GOLD. It makes a rouge wretch to think of it.

Iesha noticed the lay priest was totally lying about everything with her wicked awesome old lady perception skilz. However that information was disclosed after Raest and Bran had disappeared into the hovel/church.

Lucky Lutz, the super awesome leader of the group, had wisely decided to keep his ear to the door and therefore heard screams from within. First through the door and down the stairs Lutz sprinted only to be held up by the barbarian’s massive size in the stairwell. Bran had been stopped by…

The Tiny Thieves! They were clearly automatons (explaining why they could not be put to sleep) and surprisingly strong as they managed to subdue the brick shit house that is Bran and even Lutz: the slipperiest rouge in the land. Unfortunately for those wee jerks, Torment has a bewildering talent with his net. He effortlessly swept them off of Lutz from 10 feet away!
But what of Raest and his terrible ability to estimate bribes? He was in another corner of the savage basement of heck getting all perforated by the lay priest of a bad guy god. That’s right: Raest had paid 5 gp for an invitation to an underground murder party.

Raest cast a quick command spell which, as you may know, can be only one word. That word was “retch.” Cult-y McJerk Face then heaved up his lunch while getting pulverized via mace to the face. He died fast in a bewildering flurry of vomit and steel. Despite this early success Raest was lit up like a Christmas tree by a bunch of magic bolts from a pair of magic jerks. Torment barely missed falling into a pit trap and then charged the head cult guy who’s name probably starts with a V. Sadly V(icky?) introduced Torment to the Xmas tree experience mortally wounding the paladin. Torment collapsed to the ground unconscious and bleeding out. Bran and Raest kicked ass for awhile but succumbed to the combined forces of the tiny thieves and dickhead magic users. Low on health, they were put into a magical sleep and were soon to be easy pickings. Surely Lutz could have saved the day had he not been magically blinded and therefore unable to help his friend Torment dying just a few feet away. Aida was upstairs drinking water and that other guy was investigating the latrine. Really. They may or may not have had a slap fight while getting down the stairs.

Things looked dire for our rag tag group of misfits until the cult leader made a mistake. Which was good because the only thing stopping his dance of diabolical destruction was the late arriving, but very hydrated, team of Aida and wild magic guy. Did V(oldemort?) over estimate the strength of his magic portal’s vacuum power? Or did he simply slip like the jackass he was and always will be? Maybe we’ll find out next time because everyone but the old bat and a mostly dead Torment got sucked through the portal into another land.

Another land of suck. 5 seconds of being in the new place revealed 2 things:
1) The new bad guys are plentiful and way into lacquer armor.
2) Lutz and company are to be offerings of some sort. Which doesn’t sound good at all.

Cliffhanger!

Microservices vs David Heinemeier Hansson

Thu, 30 Apr 2015 13:13:00 +0000

Hearing that David Heinemeier Hansson had slammed microservices in his Rails Conf 2015 keynote was a bit disorienting as I was developing a microservice at the time. I leaned over to the other senior developer and said: “What’s his deal?” Toby Tripp, who had just watched the keynote, explained that when David’s team implemented microservices they had terrible trouble managing all the versions between 3 systems. “Isn’t that why we use a lookup call to get the URI-map?” I asked.

“Yep” he said.

“That and never remove a return key/value (pair), only add new ones until you can prove they are not used anymore” He added.

I returned to adding in some caching to our service calls.

Later, I watched the video which is here:

https://www.youtube.com/watch?v=oMlX9i9Icno

David has a meandering style but he mostly discusses microservices between 43:00 and 52:00. The “what’s in your backpack” stuff is all about his earlier zombie apocalypse metaphor that you can watch if you like. He claims his team implemented microservices right, loved them on day 5 but hated ‘em in year 2 (paraphrase). I don’t understand how I will regret my microservice architecture decision if we implement HATEOAS (Hypermedia as the Engine of Application State). You may remember HATEOAS as the thing from REST that all the eggheads complained was missing from Rails’ “RESTful-routes.” “Sure, the routes were REST but what about the discoverable URI-map?” they may have said. They were, of course, right (jerks).

I’m not saying HATEOAS is perfect. I was in the middle of adding LOTS of caching to our client so it almost never has to call our service for one of the two calls you must, at a minimum, make with a HATEOAS service. Why two calls?

To get the URI-map (A map of how to call the service for all resources)
Actually make the call (Using the above map)
What? There’s a 3? Yup, that second call could return another URI-map
It could continue

I know this seems insane, to make 2 or more http calls for every one request, but the benefits outweigh the costs. Old calls will always work. Less bugs. More confident software. Also, you can cache the hell out of all the URI map requests from the client side. When you need to expire that cache you can do it upon release of new functionality because, this being a microservice, you own all sides of the interface. Just work it into your deploy script and you’re done. Extra bonus: No URI ever has a stupid v7 in it.

The last call is a bit of a bummer over http. Even servers right next to each other in the cage have important amounts of latency for many services. HATEOAS isn’t for every service. Although, with virtual machines, I run my service on the same physical box in another vm: Somewhat mitigating network latency. Double Extra Bonus: My service is purely functional so I can cache all of its responses forever. That bold claim, however, is another blog post.

Anyway, Tripp is leaving my team to go be the king of software at some company I refuse to promote out of spite. How would you like to work at Backstop Solutions? I’m looking for a senior Ruby dev with a bunch of Clojure experience but, to be honest, I don’t really expect to find one. So if you’ve got some decent Ruby skills and can learn Clojure while pair programming you should apply. https://www.backstopsolutions.com/careers

Mention you read this article in the interview and you’ll get to see a man in a crazy shirt blush.

Why Group Texts Must Die

Wed, 17 Sep 2014 00:46:00 +0000

Recently I re-tweeted this thought from Pete Holmes:

top reasons for divorce. 1984: 1. financial problems. 2. religious differences. 2014: 1. constant inclusion in group texts. 2. no wifi.
— Pete Holmes (@peteholmes) September 14, 2014

because I’m in the middle of a communication crisis. I do “knowledge work” so it’s considered inappropriate to have a device constantly making little beeps and boops while the person next to me is working on some insane LibXML (you don’t wanna know) bug. Programming requires extreme concentration and distractions are to be avoided.

The rub of it is that I’m “on call” if our software product bursts into flames. Therefore, I must keep my phone in a mode capable of disturbing me so I don’t eat lunch, play ping-pong, or just code right through the disastrous failure of our software.

This used to be fine back when text messages were either:

Time sensitive
Important
From one person

In first two cases the beeps my phone made were appropriate and in the last case I could simply text back to that person that I was busy if they didn’t get the me ignoring them hint. However, as we all sadly know, the plague of group texts has completely disrupted how we handle notifications. Now when my mother in-law, a nice person who likes to share, sends me and 10 other contacts a pic of a lake I spend the rest of the afternoon getting “Nice!,” “Did you take a swim?,” “No, it’s too cold”, “where are you guys?” etc. texts. Every one of those I should probably check to make sure it isn’t an important work thing — just in case.

Why not a mass email? Why do I get group texts trying to plan something days or weeks away? Shouldn’t that be an email thread? The answer is as obvious as it is sad-making: People have given up on email. For every Inbox Zero zealot, there are a hundred people who’ve essentially let their inbox run wild with clutter. Missed important emails because of that clutter? — better move to text messages For Everything. Of course, this leads to cluttered text messages. Plenty of times I’ve been furiously searching through my email to find someone’s reply only to remember “Oh, they’re one of those text-y people, I better scroll through all the various group texts threads they’ve been involved in… Sigh.”

A small aside: Some people don’t have unlimited text messages on their phone. The horror, right? My boss consistently crashes through her text message limit b/c of “friendly” group messages.

The problem of communication clutter isn’t going away. Ever. Abandoning email for texts just moves the problem. I implore you to take charge of your inbox, people. Getting emails you never read? Unsubscribe or Block! Can’t bring yourself to do that? Well, how about creating a rule that moves your daily…

Hold on, got a text… yup, a silly friendly chatty one from a friend who never emails anymore. I really like this guy but such frivolity is a tweet or an email or a Facebook or a Whatever but not a text that demands my attention.

Where was I… Oh yeah, create rules in your email client of choice that automatically move “less often read” messages to a folder for later (read: never) reading. If you’re still getting a flood then you must keep going. Unsubscribing, blocking, and rules are your new watchwords. If you can’t handle this fire hose of information now what do you think is going happen when everyone can send “animated emojis” on a whim from their Apple Watch (or some other wearable computer thing)?

Your “inbox” is not just your email inbox. It is Every Damn Message you receive in Any form. They all via for your attention and giving up on any one communication form only annoys your friends while providing, at best, temporary relief.

Apology: I’m sorry if you’ve sent me a group text in the past and see this as an attack. I really do want to hear about your cats, see your kid’s pics, and ponder your stray thoughts… In an appropriate medium. Does that sound bitchy?

Software Apprenticeship Podcast Update

Fri, 13 Jun 2014 15:35:00 +0000

Episode 4 “Time to Exercise!” is out right now! Search for it on your favorite podcast app or check out our free temp website here: http://softwareapprenticeship.libsyn.com

With 4 weeks under his belt (plus 9 weeks of Dev Bootcamp) our apprentice, Jonathan Howden, continues his quest to become an enterprise software developer at an amazingly rapid pace. Can a dedicated man become a good developer without a college degree? Tune in and find out (spoiler: he’s doing well but it’s intense)

Topics this week:

Doing push-ups to break up the lethargy of coding
Migrating from Authlogic to Devise/Warden and the perils of using a framework’s column in the database for activation.
Why senior programmers avoid becoming mentors
Rails’ Asset Pipeline
The usual screwing around and one censored F-bomb (sorry - it was me).

Yesterday we all sat in a room and reviewed Jon’s chess code (his outside of work coding project). I’ll try to put up a more detailed article about it soon, but in brief it went well. It’s fun to watch a junior developer try to encode all the crazy logic of chess while keeping the code clean, tested, and understandable. Other than making the classic mistake of mocking/stubbing the very object he was testing, Jon has some pretty readable code that will “mostly” let 2 people play chess against each other.

Episode 5’s in the can (on the SSD?) and we’re recording episode 6 later today with Dave Hoover, co-founder of Dev Bootcamp. Jonathan attended Chicago’s DBC in September of 2013 so Dave will get to check in with how Jon is doing in the “wild.” Also, Dave and I worked at both ThoughtWorks and Obtiva together so it should be quite an interesting conversation.

I really wanted this to be a weekly podcast but here we are at the end of the 9th week and only recording our 6th ep. Now that we’ve been through the recording and editing process a few times it should be easier to stick to a weekly schedule. Even if we weren’t recording the conversation I would still have a weekly wrap up with members of the team and the apprentice — it is a very nice way to sum up the week and lessons learned. It’s sort of a weekly, recorded, low stakes retrospective.

The views and opinions expressed here are my own and don’t necessarily represent positions, strategies, or opinions of Backstop Solutions Group.

The Definition of Garbage

Tue, 10 Jun 2014 03:53:00 +0000

The views and opinions expressed here are my own and don’t necessarily represent positions, strategies, or opinions of Backstop Solutions Group.

Recently we released episode 3 of the Software Apprenticeship Podcast but had to pull it back for re-editing because of some problems with how developers talk to each other. Developers are not kind to ANY code. Even our own. Especially our own. Sitting next to a dev while he or she discusses the code they are working on can be a shocking experience. Words like “Crap,” “Junk”, “Garbage” and many worse are used often. A lot of this type of talk was on episode 3 and when someone at Backstop (who’s job it is to protect us from ourselves and comments taken out of context) heard it they asked us to edit the podcast to take out some of the more offensive comments. This is why episode 3 sometimes fades into music and then comes back mid-conversation. Sorry about that.

I don’t know where I first heard the definition of developer as “Whiny Optimist” but it is uncannily accurate. We developers are forever complaining about previously written code. Code is awful. Code is crap. Code is the worst spaghetti wrapped around horse manure we’ve ever seen.

And yet…

We couldn’t go on if we thought we’d have to live out our lives fighting the very thing we create. There is this optimism about future code. It will be bright and shiny. The next project to re-write the is going to make everything better. So much better… The code will be pristine and new features slide in like rum into coke. Ponies and rainbows are coming.

Also…

Every year I get better at what I do, so even code I thought wonderful 3 years ago can be “crap” to me today. I look back and see a developer who didn’t keep orthogonal concepts separate who coupled code that should not be coupled and I am sad. I regret my past inefficiencies and curse them.

But…

How bad is this code really? Backstop’s code is rigorously tested many times automatically before being poured over by humans. Any code change in my product gets tested first on my machine (by automated tests) then on another “Build Server” (which runs the tests I was supposed to run and a bunch more), then another series of “Regression Servers” will run some even longer regression tests that literally use the app as our customers do. If it passes all that then we’ll have our Quality Assurance people go over it again to make sure the machines haven’t missed anything. The last thing the Q.A. people do is write a new automated regression test to make sure this functionality doesn’t break in the future.

What the heck are we complaining about then? The software works! It helps many people make a lot of money, it makes the company money, and is a leader in the industry. We developers are, in some ways, a bunch of ungrateful jerks.

Let me see if I can explain why. Writing software that solves hard problems is hard. Duh. There are only so many people who can do it and we struggle through. Writing software that solves hard problems and can continue to accept new features easily is the HOLY GRAIL of software development. Rarely has it been done even though every company claims their code is the “best in the industry.” If you were to get your hands on the unedited version of episode 3 you would hear a lot of developers complain how we wish we had written code in the past that could be easily changed today. We might even call such code “garbage” but what does “garbage” really mean? In our app it has come to mean code that works, is well tested, but resists change more than we would like. We are whining about having to do more work. If only our past selves had properly separated the concerns more, if only there was more time for refactoring. But some day we will reach that shining castle on the hill. And there will be ponies and rainbows for all.

Software Apprenticeship Podcast Episode 3 is Out

Thu, 15 May 2014 03:54:00 +0000

This week we start off by throwing Jonathan into the deep end of pool where he pairs with an experienced developer on a 10 year old Java project that is the core of our signature product: Backstop. Of course the company is called Backstop Solutions and so, in order to avoid confusion, we gave the project a different name for internal use: Fund Butter. The mystery of how such a terrible thing came to pass is revealed in this very episode.

There’s no way we couldn’t discuss DHH’s Rails Conf declaration and blog post: “TDD is dead. Long live testing.” This, of course, leads to a discussion of our team’s test philosophy.

You can find it on iTunes (search for Software Apprenticeship Podcast), any podcast app’s search function, Google, this temp page: https://itunes.apple.com/us/podcast/software-apprenticeship-podcast/id868371146 or use the RSS feed directly if your into that sort of thing: http://softwareapprenticeship.libsyn.com/rss

Our panel (composed of Toby Tripp, Matt Pyra, Eric Johnson, Jonathan Howden, and I) meander through quite a few topics. Here’s a breakdown by time of the various topics:

01:27 Backstop’s 2 day bug bash
02:00 The apprentice has to tackle a 10 year old Java program
06:22 “Everything needs to have an ID if you’re going to make it Hibernate-y” - Jake “super technical” Scruggs
08:21 Why we call Backstop “Fund Butter” within the company
10:50 The apprentice encounters Selenium
12:42 Troubles with regression/integration testing through the web browser.
13:43 “Unit Testing is Dead” - DHH
20:00 Pairing all the time vs code review
21:51 Toby talks about the Hill Air Force Base pair programming study mentioned here: http://www.informit.com/articles/article.aspx?p=29410
26:40 The Wall of Awesome - Backstop’s way for employee’s to recognize and thank other employees
47:21 The anti-college movement
49:36 The “Expose your ignorance” apprenticeship pattern with examples/confessions from Jonathan, Jake, and Toby
51:14 The C3 project comes up with near 100% frequency when >= 2 die hard XP’ers are in the same room.

Announcing the Software Apprenticeship Podcast

Tue, 29 Apr 2014 01:52:00 +0000

In the summer of 2004 I did an apprenticeship of sorts at a place called Object Mentor. At the time “Uncle” Bob Martin and his son Micah Martin were in leadership positions at the company and I somehow convinced them to let me work for free over the summer in exchange for teaching me software development. It wasn’t a very structured program, nothing like what Micah would later put together for 8th Light, but I was a pretty motivated learner. I also had the advantage of coming from a teaching background so I knew how to learn.

All this has been covered in daily detail, if you’d like to read more.

After ten years of software experience I’m becoming a mentor to an apprentice and documenting the experience via podcast. Backstop Solutions has graciously allowed me to pay our apprentice (the same rate we pay interns) as he is doing real work on a daily basis in addition to outside learning experiences. From 9-5 he will be working on production code with 100% supervision as he will always be pairing with an experienced developer. It’s a six month program with 2 month check-ins.

The apprentice, Jonathan Howden, knows that should he fail to meet expectations we may end our relationship at 2, 4, or 6 months. This is a bit scary, but if Backstop is going to be taking a chance on an un-credentialed employee we, as a company, need to be able mitigate the risk of such a person polluting our codebase. It is therefore our responsibility to provide constant feedback to the apprentice so that he will know exactly what is needed to succeed. So far he’s been doing great: He just finished his 3rd week of apprenticeship so we just recorded our third podcast and will be on a weekly schedule. Assuming he survives the six month apprenticeship, Jonathan will be offered a full time job at a damn good starting salary. Interested in such a job right now? Check out https://careers.backstopsolutions.com/

In the first episode, Jonathan talks quite a bit about Dev Bootcamp (DBC). I’ve known, worked with, and read the book of one of the founders so it seemed natural to reach out to Dave Hoover and DBC to help Backstop find its first apprentice. We asked their “fairy job mother” to spread the word that we were looking for apprentices and ten applied. They were all given coding homework challenges which were evaluated code review style with the whole InvestorBridge team allowed to attend. We judged three submissions good enough to warrant an in-person interview. Jonathan made it through this gauntlet and was rewarded with a brand new, much longer, gauntlet. Of learning. Look, there’s no way not to make this sound hokey as we’re trying to do a good thing here.

On a weekly basis I hope to capture what an apprenticeship is from the inside and perhaps provide some value to proto-developers and prospective mentor companies who may be wondering what this “apprenticeship” business is all about. Changing careers is, like it or not, the future. I did it in 2004 and I hope Jonathan will too in 2014.

Software Apprenticeship Podcast: iTunes Page: https://itunes.apple.com/us/podcast/software-apprenticeship-podcast/id868371146?mt=2 RSS feed: http://softwareapprenticeship.libsyn.com/rss

SICP Wasn’t Written for You

Fri, 28 Feb 2014 03:38:00 +0000

The number of software luminaries who sing the praises of “Structure and Interpretation of Computer Programs” (referred to as SICP) is such a long list that you might think only a crazy person would take issue with it. However, to ignore SICP’s problems and continue to blindly recommend it seems just as crazy.

SICP was the textbook for MIT’s introductory programming class and was a bit of a departure from other into to computer science textbooks at the time. Wikipedia sums it up nicely: “Before SICP, the introductory courses were almost always filled with learning the details of some programming language, while SICP focuses on finding general patterns from specific problems and building software tools that embody each pattern.” Which sounds awesome, but does essentially say that abstract principles will be introduced before the nuts and bolts of a language. If you think about that for a minute, you may see where the problems will be.

When I was training to be a teacher I took a bunch of education courses. I got good grades but when I got into the classroom to actually teach I flailed around just trying to keep the class under control and mostly forgot to apply the principles I had learned. The knowledge was in my head, but it floated, disconnected, from anything in particular. When I learned these ideas I had no teaching experience, and so, nowhere to place these abstract principles.

SICP’s first chapter explains the basic form of Scheme (a Lisp), some basic operators (+, -, *, /, etc), defining/calling a function, different ways a compiler might evaluate code, and conditionals over the course of a few short pages. That’s a bit much to swallow all at once, especially the comparative evaluation stuff but that should be easily sorted out with some examples. Right? Well, that’s not really SICP’s thing. SICP will give you a few trivial examples and then toss you right into the deep end. Then first 2 problems for the reader are pretty easy, but it’s the 3rd that will let you know what yer in for: “Define a procedure that takes three numbers as arguments and returns the sum of the squares of the two larger numbers.” Which seems pretty easy until you realize there are no variables. You’ll need to figure out an algorithm that can take 3 numbers and, without any intermediate state storage, return the 2 biggest numbers in such a way that you can sum their squares. I’ll be real honest here, after about 30 min of trying to do this (I have zero functional background so I’m a complete novice here) I gave up and tracked down the answer online. Of course the answer was simple and concise and made me feel like a chump. Which is fine, but not really what I was expecting in the first chapter, let alone the 3rd problem of the entire book.

But that’s what SICP is all about – challenging problems. The rest of the chapter introduces Newton’s method for square/cube roots and lexical scoping just for fun. Chapter 2 is recursion vs iteration in terms of execution speed, resource usage, and transforming from one to the other. Logarithmic, linear, and exponential growth are dealt with in a few paragraphs and then we’re off to Exponentiation, Greatest Common Divisors, Primality, and implementing Fermat’s Little Theorem for probabilistic prime determination. My favorite question from chapter 2 asks the reader to formulate an inductive proof that Fib(n) is the closet integer to ((golden ratio)^n)/5.

Which brings me to another criticism of SICP: It assumes a familiarity with math that most people just don’t have. A first year MIT student would probably be swimming in math classes so the book assumes that knowledge on the readers part. Abstract programming principles can be very difficult to find examples for so I’m sympathetic to the plight of the authors, but when you just go straight at math you’re explaining an abstract thing with another abstract thing.

There’s a certain sort of person who gets excited by complicated abstract but internally consistent logic with no real connection to the concrete. In my experience as a physics teacher, these students do exist but are very rare. Most people need a bit of connection to something tangible in order to have the ideas connect in their brain.

What then is my point about SICP? Simply that its explanations are overly terse and its problems are large steps past what little is explained. In light of those things I have recommendations for those who attempt to work through it.

If you intend to do every problem, realize that this will take a LONG time and involve a bunch of research.
Set a time-box for how long you’re going to spend on a problem before you go look up the answer. If you’ve spent enough time trying to solve a problem you will still value the answer enough to remember it. 30 min is a good number. Increase or decrease as your sanity allows.
If you feel like something hasn’t been explained: You’re probably right. After you find the answer, a close re-reading will reveal a cryptic sentence that you now realize was trying to tell you something. This will infuriate you and is perfectly normal.
Work through the book with a group. This will hopefully allow you to commiserate about how lost you are and get some help. If there’s someone in there that loves this book and thinks everything is explained perfectly, ignore them. If they subtly imply that you’re stupid for not getting it: Leave the group. You don’t need that static in your life.
Do not feel bad about not knowing all this math stuff: Remember that this book was written for students who would be surrounded by math at the time they read it.
Consider learning Lisp before starting this book. The really important concepts in the book come easier if you’re not also learning Lisp at the same time

Why I abandoned MetricFu

Thu, 09 Aug 2012 01:16:00 +0000

My last commit to MetricFu was March 2… of 2011.

So. much. shame.

Since I am throwing in the towel and handing over leadership (perhaps to you), I thought I should explain why I’ve been such a terrible maintainer and why I’m walking away.

Reasons:

MetricFu is a mess. At some point, with all the submissions coming in, the architecture and testing got away from me. Now it’s pretty depressing to see crap code quality in an app that is supposed to be about good code quality. I can not overstate how embarrassing and frustrating this is – mostly because it’s my fault (I let in things I shouldn’t have. Stupid feature blindness).
In 2008, when I created MetricFu, I was fascinated by metrics and now I’m not because I don’t need the tools to see the problems anymore. Metrics made me better but now I’m leaving them behind: Selfish.
I’m not a consultant at the moment so the exploratory nature of MetricFu isn’t very useful to me. I know where the bad parts of town are in my app.
I think metrics can be a great teaching tool, but you can learn all the wrong things if you’re not careful. You can reduce Flog scores without making the app better (Simple: arbitrarily divide up every large method) . Same with code coverage. Even code duplication detection can lead to bad code. DRY-ing things up in a bad way can often be much more harmful then leaving them moist (perhaps you removed some duplication by creating a crazy amount of indirection – a common problem).
Ultimately, creating software is a craft. It’s not pure science and it’s not pure art, it’s a combination of both. MetricFu leans a too little hard toward numbers and that implies a precision we just don’t have. How to get from bad code to good is a very delicate dance between content, form, and business requirements.
Integrating over a dozen meta-code gems is a god damn pain. The gems that make up metric_fu use code to analyze code (intense meta Ruby) and so they sometimes conflict. Also since MetricFu mostly uses regexes to parse the (command line or html) output, every change in one of MetricFu’s underlying gems means that MetricFu needs a version bump.
The last year has been, personally, very busy and difficult for me.

For these reasons I’m going to be stepping aside as metric_fu maintainer – I’ll still be around to answer questions and such but I probably won’t be contributing much code. If I’ve somehow disappointed or frustrated you because of last year’s neglect then I apologize. Seriously. I’m sorry.

Want to lead MetricFu? Make your case in the metric_fu google group: https://groups.google.com/forum/?fromgroups#!forum/metric_fu

Oracle Foreign Key without Index Test

Tue, 07 Dec 2010 15:50:00 +0000

We’ve been having some Oracle deadlock issues that have been hard to reproduce locally. After a lot of investigation and solving of important problems that happened not to be THE problem we figured out that, while we’ve been pretty good creating integrity constraints in the database, we have not been good about making sure that every foreign key has a corresponding index. And that can lead to problems.

So we had a situation were our documents table had a foreign key on the accounts table that was not indexed. So updating an account row lead to a whole table lock on documents (instead of just a row lock which would have happened if there was an index) and that was very bad when we had two separate processes where one was doing a bunch of accounts stuff and the other was doing a lot of documents stuff. Deadlocks for everyone!

The sad thing is that if we had just drank the Rails cool-aid about having no integrity constraints in the db we would have been fine but we got into trouble by only implementing constraints and not the indexes they work much better with.

So we found this cool bit of SQL that helped us find all the foreign keys missing an index here: http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:4530093713805#26568859366976

And that help us solve the deadlock problem. But what about the future? If only there was a way to run this check periodically and automatically… Time for a test!

As an additional bonus, the failing output of the test tells you how to write the index you need. Big thanks to Dave Bortz for tracking down this problem – I just swooped in and wrote the test. And took credit in this here blog.

Ruby Kaigi 2010 Day 3

Sun, 29 Aug 2010 04:09:00 +0000

Last day of Ruby Kaigi! Sad to see it go, it’s been a great conference. As per usual Tweets are in bulleted italics and the rest is after the fact commentary.

First I’ve got to show you the commemorative fans they were handing out:

It’s Matz and… Someone else (sorry if it’s obvious - I don’t know). And of course they are programing in the bath.

BigDecimal: You can handle numbers as large as can fit into memory as opposed to the IEEE double #rubykaigi
BigMath is the Math module for BigDecimal #rubykaigi
Lots of different rounding modes in BigDecimal #rubykaigi
BigDecimal.mode is global per process - not thread safe #rubykaigi
So BigDecimal is Fiber unsafe #rubykaigi

Bummer.

.@mrkn implemented a solution storing mode in thread local #rubykaigi

Oh good. Wait, is that in 1.9.2 or trunk. I’d have to be in 1.9.2 right? To the Google! … Yep, seems like it made it in: http://redmine.ruby-lang.org/issues/show/3285

You can now change modes in a block (added to trunk yesterday) #rubykaigi
Effective digits/ Significant digits determines which digits to keep and which to round off #rubykaigi
BigDecimals don’t know their own effective digits #rubykaigi

But I think the point of talking about sig figs was that it’s coming soonish in Ruby. 1.9.3?

It is dangerous to mix floats and BigDecimals #rubykaigi
BigDecimals can’t convert from rational, integer, or float. just strings #rubykaigi
And you can’t do BigDecimal(BigDecimal) !! #rubykaigi

That is one crappy interface. Taking in only strings is frustrating and weird. How did this happen?

Now a preview of the future of BigDecimal #rubykaigi
No library for irrational numbers in Ruby #rubykaigi
Perhaps we can represent irrational numbers as algorithms and on convert when needed. #rubykaigi
BigDecimals don’t have real significant digits implemented right now. #rubykaigi
The issue of BigDecimal not being able to handle anything other than a string as input will be fixed as it is a bug #rubykaigi

The irrational numbers thing sounds pretty cool. Since I mostly sling around strings for my day job I don’t know that I’ll ever use it. But purely for Ruby pride I like to see ruby challenge Python’s rising dominance in the sciences and maths.

I really don’t like the lack of travel time between session. You should have at least 5 minutes to change sessions. #rubykaigi

Look, this has been a great conference and any slights I’ve tended to overlook because they’ve really done a heroic job of keeping the admission price down but that lack of a passing period is just… Well I don’t see why they did it. Just having the time as a buffer in case talks run over is reason alone. Moreover, no passing period traps people in sessions that they’d rather not be in and restricts choice.

Apparently there is a lot of chatter in IRC by ruby commiters that NArray should be added to strd lib. #rubykaigi
NArray is 28x faster and uses 8x less characters in 1.9.2 #rubykaigi
NArray is faster than what? Array, I assume, but I may not be following the translation correctly. #rubykaigi

Translation is volunteer and best-effort so it adds some challenge to attending sessions in a foreign language. So if I make any mistakes in this here blog, that’s what I’m blaming it on.

Pwrake is parallel distributed Rake - being developed here in Tsukuba. #rubykaigi

Just a little tidbit dropped at the end of the NArray presentation – sounds pretty cool. Masahiro Tanaka is using it drive his workflow in his scientific research.

yarv2llvm tries to speed up Ruby by implementing type inference (yarv is the vm for 1.9x) #rubykaigi
There are times in Ruby when type can not be inferred and in those cases yarv2llvm is often slower than Ruby 1.9x #rubykaigi
Fixnum overflow to BigNum is one of the hardest problems to solve #rubykaigi
Btw, I think the title of this talk: “How Did Yarv2llvm Fail” is wonderful. Way to keep everything above board. #rubykaigi

Great talk and very honest. The type inference thing looks promising but there are a couple of tough hurdles to clear. Maybe some day. Until then there’s always Mirah.

Ruby AOT complier is “mostly compatible” with Ruby 1.9 #rubykaigi
AOT == “Ahead of time”
Ruby AOT compiler is passing 7847 of 7850 Ruby Spec tests as of now. #rubykaigi
On average Ruby AOT compiler is 3.5x faster than 1.9 for common benchmarks #rubykaigi

Cool, let’s all move to Ruby AOT!

It doesn’t fair quite so well in the real world Ex: Rails. Slightly slower than 1.9 #rubykaigi
Something about cache misses makes it slower #rubykaigi
The Ruby AOT compiler team wants to reduce the generated code size to increase speed. #rubykaigi
Rails can be compiled in 77 min and size of compiled code is 92MB with Ruby AOT #rubykaigi

Oh. Never mind. Plus, compiling. Boo. Hiss.

This memory profiler’s gui looks amazing! #rubykaigi
Seriously, this is the sexiest profiler ever #rubykaigi
Can attach to running Ruby programs from another machine #rubykaigi
Designed to have as small as possible impact and be easy to use #rubykaigi

I gotta tell ya the audience was chomping at the bit to get a hold of this profiler. You can run it in production for christ’s sake! And the GUI is to die for.

Not available yet!?! #rubykaigi
Needs to get patch accepted into ruby core - maybe in 1.9.3. RATS! #rubykaigi

Very cruel of Tetsu Soh not to mention this up front. I’m sure it was an oversight but everyone was crazy disappointed. Still this was one of the best talks of the conf. Tetsu Soh is one to watch.

Dear Ruby Core please, please, please, please, please, please, please, please accept Tetsu Soh memory profiler patch #rubykaigi

Indeed. I’m famous (on the internet (in one corner)) so you should listen to me.

Lots of requests to publish Tetsu Soh’s memory profiler on Github. I agree! #rubykaigi
Automatic sliding doors in Japan trigger much latter than Americans would expect leading to a lot of stopping and hand waving

I’m not the only one who noticed this so I’m not crazy.

More awesome techno in the #rubykaigi main hall. I want, nay, need the playlist.

Please post it on the ‘Goodies’ section of the Ruby Kaigi site. Or tweet about it. Something.

Shay Friedman has spent the first five minutes of his talk apologizing for being associated with Microsoft. Stop. #rubykaigi

There was this weird hostile vibe coming from Shay. I felt like he hated us for liking Apple products. The weird thing is that there are a lot fewer Apple computers here than at a normal Ruby Conf. Maybe he thought we were all Microsoft haters and so he might as well fire first. Seemed like a mistake.

Just saw a nice hello world creating windows and dialog boxes in IronRuby #rubykaigi
IronRuby currently passes 85% of Ruby Spec. #rubykaigi
IronRuby 1.0 is 1.8.6 compatible. 1.1 (coming soon) will be 1.9.2 compatible. #rubykaigi
Windows Presentation Foundation’s view templates are written in xaml which is like html. #rubykaigi

Another markup language because we needed more. I feel like the IronRuby team really needs a win. They’ve been the slowest progressing Ruby VM for a long time now.

Pretty neat trick that. Of course it means making your site dependent on silverlight. No worse than depending on Flash I guess.

IronRuby.Rack is Rack implementation on ASP. Currently in beta. #rubykaigi

It’s all coming along, I’d just like to see something working at this point in its life.

If key is symbol then only one instance - less object creation. Which is why everyone uses symbols. Or because everyone else does #rubykaigi

So did you know why everyone uses symbols? Or did you just do it because that’s the convention. Yeah, me too but then I learned. I’m not going to say how long ago that was.

Presenter’s computer went down and it’s taking forever to reboot. Stupid Mac. #rubykaigi

That’s one of those rough moments where you really feel bad for the presentor. But he handled it like a champ – continuing the presentation while he waited for his machine to boot back up. Well done. I wish I got more out of the talk but it was one of those inspirational talks that are hard to translate. The Japanese speaking audience seemed to love it.

Please use the overburdened network to download the english version of the slides. Umm… What? #rubykaigi

Huh. Why did that seem like a good idea. Since when does asking a room full of people to download something all at once ever work. And yet, it seems it happens once a conference. Repeat after me: Never ever depend on the network at a conference.

Funny story time. All through the conference I spent a lot of time staring at the IRC screens on either side of the stage where the translations happen. But since it’s just IRC, anyone can join the room. There was this one guy who posted A LOT and everyone of the posts seemed to be either:

888888888888

wwwwwwwwww

I later found out that 888888 means clapping and wwwww means laughter. OK but stop cluttering up the screen that I’m trying to read translations off of. Then I found out who sora_h was: He’s 14 and his name is Shota Fukumori. He got up and gave an entertaining lightning talk. Turns out he’s a Ruby commiter so he’s got that on me.

And, all of a sudden, it was time for the final keynote by Chad Fowler. He gave a talk about how to live a remarkable life and it was good stuff.

The two things that are necessary to live a remarkable life are: To have intention and a system of realizing that intention #rubykaigi
Living intentionally means being mindful of your actions. It’s easy to coast through life. #rubykaigi
“My co-workers laughed at me when I learned Ruby” - Chad Fowler #rubykaigi
“You don’t want to be in a situation where you are competing on price” - Chad Fowler #rubykaigi
Programming Cobol is like working in a hospice. There’s a market for that. - Chad Fowler #rubykaigi
“Always be the worst musician in whatever band you’re in” Pat Matheney #rubykaigi
“If you’re playing things that sound good then you’re not practicing” lesson from music that applies to programming #rubykaigi
“I needed a project that was useless” - Chad Fowler #rubykaigi
“Was today better than yesterday?” All you have to do is get a little better every day #rubykaigi
“Passion is a resource that you have to conserve” - Chad Fowler #rubykaigi

Lots of gems in there.

RubyKaigi 2011 will be in July and in Tokyo. #rubykaigi

There was something about the theme being “The Last Ruby Kaigi” but I think it was sort of a joke making fun of some guy who posted a rant about how the Ruby Kaigis need to end. I didn’t have enough background to get it.

Oh, I forgot to talk about the guy hawking “The Last Google Wave Book Ever Published.” It seems he had been working on a Google Wave book and it was printed on the same day Google announced the end of wave. His response? To shamelessly promote the book at Ruby Kaigi in a lightning talk, at some sort of hallway session (with 30 people gathered around), and pretty much everywhere else. And I actually saw some people carrying the book around. Did they buy it? Who knows – his positive personality was so powerful I wouldn’t be surprised if they did.

Ruby Kaigi was an excellent time. I thought it might be crazy intimidating but everyone was super nice and there were enough english speakers so that I could always get my ideas across. Go if you have the chance.

Also, check out my every growing set of Ruby Kaigi photos at Flickr:

http://www.flickr.com/photos/jakescruggs/sets/72157624815648014/

Ruby Kaigi 2010 Day 2

Sat, 28 Aug 2010 12:27:00 +0000

Holy crap am I tired. It’s been a long awesome day. It started out with some excitement:

Just found out I have an hour time slot when all I prepared was 30 minutes. Ok, time to write some more. #rubykaigi
Panic! Maybe I’ll talk about metric_fu a bit. #rubykaigi

I must have looked at that schedule 20 times and never realized that I had an hour slot. Everyone else had 30 minutes so I thought I did too.

Attendees of “The Importance and Implementation of Speedy Tests” will also get “Metrics Based Refactoring” at no additional cost! #rubykaigi

I did something I almost never do: Look at crisis as an opportunity (crisi-tunity). I had to write “Metrics Based Refactoring” anyway for Windy City Rails so why not write it now. In less than 4 hours. While watching my friends present at a conference. Looking over those sentences now I can’t believe I didn’t freak out.

Ted Han used publicly available data to settle reviewing bais accusations against “The Edge” of hating the PS3 #rubykaigi
Turns out there was no bais. Math to the rescue! #rubykaigi
bais? bias? baise? no idea.

You are probably not surprised to find out that I can’t spell. Even Eito Katagiri, who did a wonderful job translating my slides, found a bunch of spelling errors and English is not his first language.

I just pulled out a table in the main hall and the table police where all over me. My mistake. Sorry. #rubykaigi

There were announcements everywhere about not doing this and yet I did. In my defense: I’m an idiot.

They’re giving out an award to the person who committed most to Ruby 1.9.2: Yusuke Endoh #rubykaigi

A very nice gesture. Lots of class here at Ruby Kaigi.

Time for Matz’s keynote!

Matz is talking about Ruby 2.0… again. #rubykaigi

This topic is a staple of Matz’s speaking career. I think the first time I saw him talk, years ago, he was talking about Ruby 2.0. Someday…

“Right now ruby is just good enough” - Matz #rubykaigi
Matz hates local variable propagation (the lack thereof) but no one else seems to care so he’s abandoning it. #rubykaigi
Ruby’s private is not private: it can be accessed from subclasses and overridden by accident #rubykaigi
Monkey patching modifies the class globally. #rubykaigi
Classbox is the solution to global monkey patching #rubykaigi

Lots of talk about the mysterious ‘classbox.’ What is it? Well here’s a paper on the subject:

http://www.iam.unibe.ch/~scg/Archive/Papers/Berg05aclassboxes.pdf

And some pertinent lines from the abstract:

…Unfortunately existing approaches suffer from various limitations. Either class extensions have a global impact, with possibly negative effects for unexpected clients, or they have a purely local impact, with neg- ative results for collaborating clients. Furthermore, conflicting class extensions are either disallowed, or resolved by linearization, with consequent negative effects. To solve these problems we present classboxes, a module system for object-oriented lan- guages that provides for method addition and replacement. Moreover, the changes made by a classbox are only visible to that classbox (or classboxes that import it), a feature we call local rebinding. To validate the model we have implemented it in the Squeak Smalltalk environment, and performed benchmarks.

Now for more Ruby 2.0 preview:

5/2 => 2 should be 2.5 or 5/2 (rational) #rubykaigi
Inheritance in ruby is more for connivence than for other merits #rubykaigi
Matz is thinking about moving mix-ins to a traits like solution which would have conflict detection (unintentional overrides) #rubykaigi
You could declare the ‘mix’ and specify how to deal with conflicting methods. #rubykaigi
mix Foo, [:*] would copy all constants from the mix, or you can specify which ones you want and rename them #rubykaigi
mix raises error on method/constant name conflict or removing #rubykaigi
From Matz’s slides “Ruby 2.0, just started, small step from 1.9, should be done soon” #rubykaigi
Ruby 2.0: Traits, Classbox, Keyword arguments, a few other nifty features #rubykaigi
There some fierce discussion going on in IRC about the proposed changes in Ruby 2.0 #rubykaigi

The debate on ‘mixes’ got pretty hot and heavy. Keep in mind that all this IRC chatter is displayed behind Matz while he was giving his keynote because the translations are done in IRC. So you’d be reading a translation of what he said right along with people discussing it.

My presentation on “Speedy Tests” and “Metrics Based Refactoring” starts at 13:30 (10 min) in room 200 #rubykaigi
The ‘slides’ from my presentation on “Speedy Tests” http://is.gd/eHNTN #rubykaigi

I think the talk went very well. Especially considering that I wrote the second half fairly fast.

I just disparaged integration tests in favor of unit tests. Next up is @p_elliott talking about how he only does integration #rubykaigi
.@p_elliott does a lot of things to make his integration test fly. I would like to see one of their suites. I could learn a lot. #rubykaigi

So I asked Paul how long their suites take and he said that they develop on 8 core machines and use Specjour to utilize 4 more cores so they tend to run between 5-8 minutes. Pretty damn fast for full stack integration testing.

Hey, someone else made the 30 minutes vs. 60 minutes mistake. I feel better now. #rubykaigi

Except he found out at the end of his presentation when he asked: “How much time do I have left?” and go the response: “25 minutes.” Yipes. Luckily there were a lot of questions.

.@nusco did a really good job explaining the basics of Ruby metaprograming
RT @sudhindraRao Whatever works in #java does not work in #ruby. Even huge classes are maintainable. @rubykaigi
.@nusco’s favorite metaprogramming trick is method_missing. I thought he was kidding but he was not. #rubykaigi

When he said method_missing I grabbed the microphone back and responded: “Really?” I couldn’t help myself. I tend to avoid method_missing. There’s usually a way to do what you want with other programming tricks.

Modules are extremely decoupled and can be tested in isolation so they are very flexible @nusco #rubykaigi

Good point.

Lightning Talks! I’m excited! #rubykaigi

Talking very fast does not lend itself to translation. But still they were very cool. Even better there was a lady in a kimono who would ‘gong’ you if you ran out of time.

That is awesome.

After the lightning talks a few of us went back to the hotel to drop our things of before the party and we ran smack into this huge festival that happens once a year in Tsukuba. Here are some pics:

The party was super nice. And they had a fantastic spread of wonderful foods and drinks.

Ruby Kaigi 2010 Day 1

Fri, 27 Aug 2010 23:19:00 +0000

Here I am in Japan at RubyKaigi 2010. Wow. Generally I tweet a lot about the conf live and then publish those tweets here (in italics) and provide slightly more commentary. So lets get it on.

So my flight leaves at noon for #rubykaigi, takes 13 hours, and arrives at 3pm tomorrow… Wait – that can’t be right. #looksitupagain
Before today’s trip to Japan the furthest from the U.S.A. I’ve ever been is: Canada. #howisthatpossible
In cab. It has begun.
Someone once said “If you’ve never missed a flight you’re spending too much time in airports” #atgate2hoursearly

Yep - can you feel the panic in those first couple of tweets? I was totally freaked out. The actual trip turned out to be easy. I met up with Chad Fowler, Yehuda, and Woody at the airport and we took a bus straight to Tsukuba. I went to an exotic foreign land and the first thing I did was take a bus through 100 minutes of strip malls. But I was in Japan. And Yehuda held forth on ‘snowmen’ and encodings for most of the way so that was interesting.

So it’s 7:15 am in Chicago, meaning I’ve been up for over 24 hours. So that’s like 36 old man hours.
My first meal in Japan was near 60 bucks. And I’m not even in Tokyo yet. I’m gonna need a bigger wallet.
Also, I almost got run over by not one, but two bicyclists using cell phones to text.
And 7-11’s are everywhere.

So I survived the first day and even had an excellent dinner. It was a bit pricey but worth it. Many thanks to Makoto Inoue for helping organize this get together - it was exactly what I needed.

Donuts are cute in Japan: http://is.gd/eFXCR

Ate second breakfast/lunch at “Mr. Donut” which is really nice here in Japan. Of course you can get noodles.

Good news: #rubykaigi has a non-freezing temperature. I was worried it would be 91 outside and 50 inside. Which tends to make me sick
Some nice low-key techno playing in the #rubykaigi main convention hall.
RT @headius Ruby 1.8.8 will release this year and be the last 1.8 release. Some debate still about whether to backport 1.9.2 features. #rubykaigi

Ah, Charles got to got to the secret Ruby Core meeting so he’s in the know. Actually I could have probably gone too but I wasn’t sure if just anyone was allowed.

Btw, the advice I received to stay up as long as possible after my flight to Japan was good. Feeling no jet-lag.
I adjusted my Japanese rent-a-phone to display am/pm and now it reads “0:32pm” I guess they really like military time here

It’s the little things that are the most endearing.

Getting an introduction to #rubykaigi in japanese with translations coming in IRC which is displayed on side screens. #lag

Next up at #rubykaigi is a panel on Rails 3/ Ruby 1.9.2 (replacing canceled keynote)

Ah, the translations are back. Mostly. #rubykaigi
RT @headius I can’t decide if it will be more or less exhausting to attend three days of conference sessions I can’t understand :) #RubyKaigi
.@wycats is fearless - he’s critiquing Ruby 1.9.2 while sitting 5 feet from matz on stage

So Leonard was doing the translation from Japanese to English and Matz was doing the translation from English to Japanese which lead the a moment where Matz had to translate Yehuda’s (nice) criticism of Ruby 1.9.

One of things @tenderlove really likes about 1.9 is using encodings is painless. You have to think about it but it’s easy. #rubykaigi
Secret to getting commit rights on Ruby or Rails? Submit patches with tests over a consistent period. #rubykaigi
.@tenderlove doesn’t think that ActiveRecord got the same amount of love that ActiveSupport did in Rails 3. #rubykaigi
Specifically @tenderlove doesn’t like ActiveRelation’s integration in ActiveRecord in Rails 3. “It needs help” #rubykaigi
.@wycats’ response: “There’s always 3.1” #rubykaigi
.@tenderlove feels less able to bounce around the whole project when developing on Ruby as compared to Rails. #rubykaigi
.@wycats brought up something for Ruby core, saw a lot of discussion referencing his name but he could not participate. #rubykaigi

Ah the perils of trying to develop across (real) languages.

Wait, Sarah’s talk is going to be in Japanese? #rubykaigi
Oh, just the first part – well done. #rubykaigi

She learned a lot of Japanese just for this presentation. Good for her.

Rails wrapping of Javascript is kind of a disaster - mostly because javascript is changing fast. #rubykaigi

Good point - I hadn’t really thought about why wrapping SQL works so well while wrapping Javascript works out so poorly.

Step one to writing testable Javascript: Get it out of the view. #rubykaigi
Pivotal uses Jasmine to test their Javascript: Bdd/RSpec like syntax and no dom is required. Can run in browser or headless. #rubykaigi
There’s no time between sessions to escape one and go to another. Good thing I don’t mind appearing rude. #rubykaigi

Picture a lot of me saying “Excuse me” to people who don’t understand english.

OH “Social games are just CMS with bad UI’s” #rubykaigi

That’s really funny.

MySql 5 only supports 3 bytes for UTF8. huh. #rubykaigi
A ‘u’ with an umlaut is two code points that represent one character. #rubykaigi
UTF8 and UTF16 are both encodings of unicode. #rubykaigi
In Ruby 1.8 and C a string is just an array of bytes. #rubykaigi
“Corruption is normal” - @wycats #rubykaigi
force_encoding is not the way. If you have to use it you probably have a deeper problem. #rubykaigi
Dear internet: Take all sweeping statements with a grain of salt.

Two sessions in a row on encodings. We all feel like we need to know more about encodings. And then we ignore that feeling until it bites us in the ass.

jugyo has a lot of Growl-themed ideas. #rubykaigi
Had an outbreak of super-sleepiness. Purchased a strange energy drink from an even stranger vending machine and I’m good. #rubykaigi
TermColor can do blink! Now that’s progress. #rubykaigi
Cinatra is Sinatra for command line apps. #rubykaigi
“write code like writing blog entries” - jugyo #rubykaigi

Watching Jugyo talk is always entertaining - I loved his lightning talk at last years Ruby Conf.

.@tenderlove has changed into a crappy suit – It’s business time! #rubykaigi
And headgear? #rubykaigi

That, my friends, is awesome. I can’t compete in shear crazy and acknowledge my superior.

The number of languages (code) vs. the number of languages (speak) is completely off. #rubykaigi
“People are interpreters with forgiving parsers.” - @tenderlove #rubykaigi
.@tenderlove enjoys programming the most at hack nights. Challenging and fun. #rubykaigi
PHP and Ruby living together: Webrick serving up WordPress. You are one weird dude @tenderlove #rubykaigi

Webrick serving up PHP WordPress. Think about that for a moment.

Making some last minute changes to my #rubykaigi presentation: “Speedy Tests” Come see it tomorrow at 13:30 in room 200

Hey, I just found out I have an hour time slot when I had thought I was going to present for 30 minutes. I guess the crowd is going to get some bonus metric_fu coverage. As I’m going to present in 4 hours I better go write some more content. Panic!

Oh yeah, go check out my Ruby Kaigi photo set on Flickr: http://www.flickr.com/photos/jakescruggs/sets/72157624815648014/

Using Git Inside a Git Hook

Tue, 20 Jul 2010 20:15:00 +0000

Using Git Inside a Git Hook can cause problems. In my previous post: “Signal 13 Problems with Git Hooks” I describe how we are trying to automatically merge certain types of branches into a branch that is designed to hold them all. Anyway, that means we want to run some git commands inside of the git hook. We change dirs into another directory where we have a clone of the repo and start telling git to merge some stuff and we get a bunch of

remote: fatal: Not a git repository: ‘.’

But if we run the exact same commands as the git user everything works fine. Huh. Eventually we got our linux guru over and he noticed that the environment under which the git user runs is totally different when inside a hook. Gitolite does a bunch of things to the env, but the one that was screwing us up was the setting of the GIT_DIR. After we figured that out, the solution was as easy as:

ENV.delete ‘GIT_DIR’

in our ruby script that is triggered by the ‘post-receive’ hook. And now I must get back to the fun.

Signal 13 Problems with Git Hooks

Tue, 20 Jul 2010 17:16:00 +0000

Ran into a gotcha in Git today when trying to write a post push hook. We want our designer to have a fast turn around time with clients so we’re writing some hooks to merge all of the ‘theme’ branches he works with to get merged into a special preview branch which is then deployed to the preview site. And all this should happen after he does a ‘git push.’ Seems like a ‘post-receive’ hook is just what we want.

Except that every time we tried to create one we got these errors on a push:

error: git-shell died of signal 13
fatal: The remote end hung up unexpectedly
error: error in sideband demultiplexer

If the ‘post-receive’ file even existed in git_dir/hooks/ on the git repo box, we got this error. We checked permissions, gitolite docs, git docs, google, etc and no help. We finally realized that Git was piping in some information to our ‘post-receive’ file and since we were not consuming it, that was causing the explosion.

So, I present to you, a stub of ‘post-receive’ file written in Ruby:

#!/usr/bin/env ruby
STDIN.readlines.each do |line|
rev_old, rev_new, ref = line.split(“ “)
# You will get in here as many times as branches were pushed
end

rev_old is the old commit hash, rev_new is the new commit hash, and ref will be something like: “refs/heads/test_branch” Useful information.

Git often passes things into it’s hooks, check the git book docs to find out what.

Ruby Midwest 2010 Saturday

Sun, 18 Jul 2010 02:39:00 +0000

So day 2 begins. I got to bed early-ish as I am old so I’m fresh as a daisy and ready for more Ruby. As per my established practice, Tweets are in italics.

Keynote by Yehuda Katz (@wycats) is up next at #RubyMidwest
I’m wondering why you can’t give a technical keynote? Everyone says so. #RubyMidwest

Why is that, exactly? It’s a technical conference. And a single track one at that. So I say tech it up, baby.

Things that seem really easy are actually huge blockers to new users @wycats #RubyMidwest
.@wycats started Rails dev on windows - me too. #RubyMidwest
“What the F** is that thing with raw_host on it?” Ex of a small thing that is a blocker to a noob @wycats #RubyMidwest

Yeah, I had a few of those. I had no idea how to do public static final in Ruby. When I figured it out it lead to one of my early blog posts: Public Static Final for Ruby

RT: @RubyMidwest Big thanks to @mbleigh and @intridea for hosting the OMGWTFBBQ dinner/lightening talks/hack night! #rubymidwest
RT: @RubyMidwest great roundup of the first day of @rubymidwest from @jakescruggs http://bit.ly/arfY9m #rubymidwest (via @ajsharp)

Oh yeah – the Ruby Midwest Twitter account liked my post. Go me.

.@wycats got into the ‘subversive’ side of the Rails community through Data Mapper because he had a legacy DB #RubyMidwest
“Because of the craziness of what I was doing, Merb was much more appealing to me than Rails” @wycats #RubyMidwest
Building the Merb Server (never finished) pushed him hard into areas he didn’t understand and made him much better @wycats #RubyMidwest
RT: @tswicegood “If you’re gonna do something open-source, pick something hard.” /via @wycats #rubymidwest

Good advice. Hard to work up the nerve to follow it but good none-the-less.

Bundler has been much harder than Rails - It’s a hard social problem. @wycats #RubyMidwest

Yeah. I’m the advocate for Bundler on my Rails 2.3x project and I’ve heard a lot of “Stupid Bundler!” and it was almost never Bundler’s fault.

Strength of the Rails community: We demand a very high level of external api excellence. @wycats #RubyMidwest
“User experience includes the command line” @wycats #RubyMidwest
We’ve spent a lot of time making the command line an enjoyable experience. Not so true in other communities. @wycats #RubyMidwest
.@wycats spent a bunch of time making Bundler’s error messages more informative. #RubyMidwest

Good man. It really helps.

People were confused by “Locking gems” so that inspired 0.9 => 1.0 @wycats #RubyMidwest

Yep, I kept having to explain to my team that we lock because we’re in production.

Now, in 1.0, you get a lock automatically in Bundler. @wycats #RubyMidwest

Nice!

People have a right to expect basic use of a product to not require a manual @wycats #RubyMidwest
Oh the other hand, with advanced features you should expect the users to read the manual @wycats #RubyMidwest
Sign of a mature project: Successful maintainer hand off @wycats #RubyMidwest
Another sign of a mature project: Survives several Ruby/Rails releases @wycats #RubyMidwest
Another sign of a mature project: Incorporating real world crazy bug fixes @wycats #RubyMidwest
.@wycats Sass is battle tested while Less is not. #RubyMidwest

Oh no he di’ent! Let the flame war begin.

New things should be able to disrupt, but points should be given to things that are mature @wycats #RubyMidwest
.@wycats “Merb was massively immature when it got a lot of mind share. It should have been much harder than it was.” #RubyMidwest

Really honest of him to admit that. Also, while I’m at it I should point out that it’s really cool of him and Chris Wanstrath to show up to a small first year regional Ruby conference. Stand up guys, they are.

RT: @j_root #rubymidwest you heard it here first Yehuda Katz said that less css is clown school… and that Sass 3 rules its nuts. :) #misrepresent

It should be noted that @j_root used the hashtag #misrepresent AND a smiley emoticon so he’s just trying to stir up trouble. Bring it.

RT: @j_root #rubymidwest Yehuda spits mad fire, holding the entire oss community to the flame. you can run but you can’t hide, gems… bam pwnt

I’m not sure what “bam pwnt” means but I like the cut of this @j_root’s jib.

“User confusion is an actual bug” @wycats #RubyMidwest

Good point.

OSS users: put pressure on the established projects to incorporate the new hotness instead of immediately jumping ship @wycats #RubyMidwest
.@wycats Reminds me of Winston Wolf. He just gracefully handled a “Why were you a dick to me online” question #RubyMidwest
To be fair, I don’t think the question asker meant it to come out that way. But it kinda did. #RubyMidwest

Remember the scene in “Pulp Fiction” where the Wolf showed up and effortlessly dealt with Vince Vega’s guff? It was like that but better. Do NOT throw down with the wycats - he will destroy you.

Next up “User Experience for Library Designers” - Wesley Beary (@geemus) at #RubyMidwest
Rename! talk is now called “eXperience Driven Design” @geemus #RubyMidwest

You can’t sneak those renames by me – I’m wise to your tricks. Of course, I’ve had some of my talks renamed behind my back by organizers. I’m looking at you, Ray Hightower. But I kid the Hightower – He’s alright with me.

But seriously, I’ll cut you if you even think of renaming one of talks again. I’m kidding! Sorta.

Slide from last night’s Lightning Talks: http://is.gd/dvOJw “This is actually a simplified diagram” said the speaker #RubyMidwest

Funny stuff. If you’re a Ruby nerd. And you probably are.

Check out http://windycityrails.org/sessions for a familiar face. Spoiler: It’s Me!

What? Windy City Rails picked me to talk? Thanks guys. Especially Ray, who didn’t take those previous jokes personally.

RT: @brntbeer @geemus is killing so far with funny slides #rubymidwest
Writing a test framework is a good way to learn if nothing else. And it probably will be nothing else. @geemus #RubyMidwest
Fog has 124 versions in just over a year. Holy crap! I thought metric_fu’s 24 was pretty good over 2 years @geemus #RubyMidwest

What the what!?! 124 versions? For real real and not for play play? Damn. I am humbled.

Fog is “The Ruby cloud computing library.” btw @geemus #RubyMidwest
“Encourage Contributors” Damn Straight. He gives out T-Shirts! @geemus #RubyMidwest

124 versions and T-Shirts? This is the best open source project ever. I may even use it someday.

RT: @Arlen Wesley Beary: “When you’re writing a library, the expert, is YOU. You are trying to help the rest along.” #rubymidwest
Who wants to design a metric_fu T-Shirt? That would be cool. #RubyMidwest

Seriously – design a shirt for MetricFu and I’ll name a major release after you.

RT: @lpillow The dirty secret to project naming: spend as much time coming up with the name as coding the project. @geemus #rubymidwest

Funny.

“Practical Projects in Mongo DB” with Alex Sharp (@ajsharp) starts soon at #RubyMidwest
MongoDB is schema-less: great for rapid agile development @ajsharp #RubyMidwest
Mongo stores documents (in binary json) not rows @ajsharp #RubyMidwest
Auto-sharding is coming soon. But I’ve heard that before. @ajsharp #RubyMidwest

Pretty sure I saw a talk on Mongo a year ago that was promising Auto-sharding. Either they announced that feature way too early or it’s proving harder than they thought. Money on the later.

Mongo writes are “fire and forget” @ajsharp #RubyMidwest
No Joins or Multi doc transactions in Mongo @ajsharp #RubyMidwest

Hmm, could make like interesting.

RT: @mattyoho @blowmage #rubymidwest has been a really great conference. Major kudos to the organizers; it’s been smooth as butter.

Here here! If any organizers come to Windy City Rails I’m buying the drinks.

.@ajsharp uses Mongo to create detailed super search-able logs that help him track down bugs #RubyMidwest
“I don’t care about storing data, I care about persisting state” @ajsharp’s argument against SQL #RubyMidwest

Good quote and a good point.

“Pragmatic Guide to Git” by Travis Swicegood (@tswicegood) coming up at #RubyMidwest
It’s going to be an Intro to Git talk. @tswicegood #RubyMidwest

Intro to Git is not really my thing (been using it for a few years) but he did a good job.

RT: @ajsharp Slides for my @mongodb talk @rubymidwest http://bit.ly/bTUSGi #rubymidwest #mongodb
RT: @LuigiMontanez My slides on Civic Hacking from #rubymidwest: http://bit.ly/cDa4RQ

Slides!

Can’t say I’ve ever understood applauding people who win books at conferences. Great job being lucky! #RubyMidwest
Maybe it’s just something to do while the winners make their way to the front. Clapping for book winners, that is. #RubyMidwest

I say that at the next conference we all sit in stony silence while the lucky one slowly makes his or her way up to the front. Who’s with me?

“Interoperable Web” by Michael Bleigh (@mbleigh) next at #RubyMidwest
ostatus is a collection of web standards, like the hall of justice (superfriends) @mbleigh #RubyMidwest

Great “Super Friends” Theme to this talk. Remember the classic 70’s-80’s cartoon? Well I do because I’m old. Get off my lawn.

RT: @adamstegman Interoperable Web wins best slide of the conference. Unicorn, hearts, rainbows, leg warmers - unattainable dream. #rubymidwest

The leg warmers on the unicorn is what really made the slide for me. Why would a Unicorn need leg warmers!?! That’s just silly.

pubsubhubbub - is webhooks with standards. you can subscribe to events @mbleigh #RubyMidwest
superfeedr provides a free hub server for public stuff @mbleigh #RubyMidwest
webfinger takes this point of view: “email = identity” And hooks everything up through that @mbleigh #RubyMidwest
redfinger is a ruby wrapper for webfinger @mbleigh #RubyMidwest
Salmon - if someone comments on a blog post on an aggregator (not the blog itself) then it can show up on the blog @mbleigh #RubyMidwest
activity streams - adding verbs to rss feeds. Ex: followed, posted, favorited, closed, updated, tagged, etc. @mbleigh #RubyMidwest

God Damn does this stuff sound sexy. Everyone needs to support all of this right now. Without me doing any work.

OAuth 2.0 flexible multi-profile token authorization. @mbleigh #RubyMidwest
In OAuth 1.0 you had to go the website for auth. What if you couldn’t (no browser), or their wasn’t a website? @mbleigh #RubyMidwest
OAuth 2.0 provides 4 ways to auth: Web server, user agent, native app, and autonomous @mbleigh #RubyMidwest

Good and needed improvements.

RT: @LuigiMontanez .@mbleigh is giving an incredible presentation on the Interoperable Web at #rubymidwest – a peak into the very near future
“Start thinking in standards” to make the web one big distributed info machine. @mbleigh #RubyMidwest
Here are @mbleigh’s slides on the Interoperable Web. http://bit.ly/bZfm8r (via @LuigiMontanez)

An exciting talk. Assuming any of it gets adopted. If not we’ll look back and weep tears of regret into the pillow of despair.

So I may have to calm down a bit on the tweeting as my talk is up after the next one and I should review my presentation. #RubyMidwest

I don’t think I did, really. I’m out of control. I’m on an airplane right now so I’ll have to quit the over tweeting cold turkey. By working on my blog post. Kinda like methadone.

“memprof” by Aman Gupta (@tmm1) next at #RubyMidwest
Ruby MRI has a “stop the world” GC. @tmm1 #RubyMidwest

Boo.

More objects == Longer GC run @tmm1 #RubyMidwest
So, of course, less objects == better performance @tmm1 #RubyMidwest
Avoid leaked references (sometimes called memory leaks, but not really the right term) @tmm1 #RubyMidwest
bleak_house not only tells you what is ‘leaking’ but also where it is leaking @tmm1 #RubyMidwest

Sounds good.

RT: @chadmontplaisir @tmm1 “God was known for a long time to have memory leaks” #rubymidwest

The God project has lead to a lot of unintentional humor. And for that, we thank you God.

memprof - easy to use, no patching the vm, detailed (file/line) object contents, refs between objects, simple json output @tmm1 #RubyMidwest
RT: @samullen I feel more smarter just by listening to @tmm1 #rubymidwest

See what he did there? They bring the wit here in the heartland.

http://memprof.com/ has gotten better since I last looked at. Nice visualizations of where your memory is going @tmm1 #RubyMidwest
http://memprof.com/ is built using Mongo @tmm1 #RubyMidwest
RT: @chadmontplaisir @tmm1 Procs are a good way to memory leak because they keep all the variable references within the creation scope. #rubymidwest.
RT: @chadmontplaisir @tmm1 Take away: Looking through the trash gives you a better understanding of what your application is doing. #rubymidwest

True dat. Optimize, always optimize.

My talk is in less than 10 minutes… Panic! #RubyMidwest

Well, it was a quiet crowd. Afternoon on the second day and all. The Twitter seemed to like me and a bunch of people sought me out after the talk for more info or just to thank me. So I guess it went well. There was a moment, at the end of the talk, when I looked up from my computer to see tumble weeds rolling across the floor. I shouldn’t worry about it I suppose.

My presentation canvas: http://is.gd/dw696 for my “Speedy Tests” talk. #RubyMidwest

I used Prezi.com for my presentation. It’s a canvas based presentation system and it’s pretty cool. You put all your ideas on a big canvas., sort and arrange them, paste in some pics or whatever, and define a path through said canvas. Lots of fun to work with.

Someone (not wearing a name tag) just turned me on to this: http://is.gd/dw6lp factory_data_preloader More speed! #RubyMidwest

That guy turned out to be Kyle Ginavan. I’m totally going to try out factory_data_preloader when I get back to work on Monday.

RT: @mattpetty “test suites should be reliable, repeatable, and understandable” -@jakescruggs #rubymidwest

My favorite quote from my talk and I completely made it up on the fly. Felt good saying it.

@lpillow It was a pic of a little girl making a frowny face. Honest.

Someone though one of my images or diagrams was pornographic. I’ve looked through the presentation 20 times and I can’t see it. If you find something, let me know.

“jQuery and Rails, Sitting in a Tree” by Adam McCrea (@adamlogic) is happening now at #RubyMidwest
Neat bit about stealing non-obtrusive javascript from Rails 3 to use in Rails 2x @adamlogic #RubyMidwest

To be honest, I was a bit burnt at this point and coming down from the post talk high so I didn’t cover this one as much as I should. Sorry Adam - say hi to Joe for me.

“A New Set Of Wheels: Leveraging Ruby For System Administration” by Josh French (@joshfrench) #RubyMidwest
GrepFu looks pretty cool. Ruby wrapper for command line searching tools. http://rubygems.org/gems/grep-fu @joshfrench #RubyMidwest

Check out the GrepFu - it looks well worth investigating. Luke Pillow, a Ruby Midwest organizer, said he uses it ever day.

Well, I gotta go catch a plane back to Chicago. Thanks #RubyMidwest I had a great time and learned a ton.

And I’m on that plane right now. It’s like a conversation with my past self. Hey, past self, don’t eat that airport BBQ - they’re gonna put mayonnaise on brisket.

Too late.

I did get to sit in the airport sports bar and listen to two young ladies in interesting outfits argue about whether it was lying to omit the fact that you didn’t sleep in your hotel room last night. It took a very long time to decide.

Ruby Midwest 2010 Friday

Sat, 17 Jul 2010 04:42:00 +0000

So people often ask me why I tweet so much at conferences (113 tweets today, for example). Well, usually I’m furiously typing notes into TextMate so I can blog about it later. Now, in the post twitter world, I type those thoughts into twitter and harvest them later for my wrap-up posts. That way people can follow a conf’s progress live or wait for the recap.

So Tweets will be in italics and everything else is bonus commentary.

Generally the first day of the conference went amazingly well. Hard to believe this is their first year. Wifi was plentiful, food was decent, and the space was nice. Oh and the talks were good too.

At @RubyMidwest waiting for the fun to begin
So I think the official hash tag for @RubyMidwest is #rbmw or at least that’s what the welcome tag said. Although #RubyMidwest is popular

So there was a lot of confusion about the official Ruby Midwest Twitter hashtag. The badges said one thing, the welcome screen said another, and everyone else chose a third: #RubyMidwest. I eventually just went with the most popular. That’s about as much controversy there was on the first day so everything went pretty well.

“Open Source sucks and I’m to blame” - Chris Wanstrath at #RubyMidwest #rbmw
No one has more experience than me with running bad open source projects - @defunkt #mwrc #rubymidwest
The trick to getting better at things is Practice. @defunkt #mwrc #rubymidwest He played guitar for 10 years and never got better nopractice

Chris never got better at the guitar because he never practiced. The same applies to code. You can knock things out for years but until you work in a mindful manner you will never get better.

It’s too easy to release open source software - @defunkt #RubyMidwest #rbmw

By which I take him to mean that lots of people throw something up on GitHub, push a gem, and provide a few lines of doc before something even works.

Levels of open source Massive (Linux kernal), Big (Rails, JQuery), Medium (homebrew) Tiny (most of the rest) @defunkt #RubyMidwest #rbmw
The tiny ones tend to be all the ones we use - and they tend to be managed poorly @defunkt #RubyMidwest #rbmw

Some open source projects are managed really well, but it’s amazing how many we use on a regular basis that are not.

Always have a license file! @defunkt #RubyMidwest #rbmw
If it doesn’t install easily it sucks because that’s your first experience with the project @defunkt #RubyMidwest #rbmw
Why do we use MySQL instead of Postgres? MySQL was way easier to use early on in Rails @defunkt #RubyMidwest
JQuery no conflict mode is another example of making install easy. You could use it and Prototype super easy @defunkt #RubyMidwest
If your api is for a human, design it to be easy and don’t assume just ruby developers will be using it. @defunkt #RubyMidwest
Lack of examples is the next biggest problem in open source - people need help getting started @defunkt #RubyMidwest
Public Api means having methods you know won’t go away when you upgrade. @defunkt #RubyMidwest
man pages are important if you have a command line interface @defunkt #RubyMidwest
Set Expectations: explain what it’s for. And make sure it does that! @defunkt #RubyMidwest
Projects that aren’t maintained… OK, that happens. But say so: “Looking for a good home” note on home page @defunkt #RubyMidwest
“Too many features is a really bad thing” @defunkt #RubyMidwest
“It’s way too easy to say yes” @defunkt #RubyMidwest #rbmw
“Much harder to remove a feature than to accept it later” @defunkt #RubyMidwest

These last 2 statements are particularly applicable to metric_fu right now. There’s a bug in the way it computes Flog averages and I’m having trouble fixing it because of an undocumented “files to ignore” feature. I may have to rip it out.

Better to set up a plugin api and let people write their own extension functionality @defunkt #RubyMidwest
The lack of competition is horrible – Look at how competition from Merb made Rails better @defunkt #RubyMidwest
Too few releases is horrible and wastes people time - Too many can be annoying too. hmm. @defunkt #RubyMidwest
Have a decent change log - super important for long time users. @defunkt #RubyMidwest
Arbitrary version numbers confuse people. Look up Semantic Versioning @defunkt #RubyMidwest
If you’re using it in production it IS 1.0 - We have a lot of numbers and we’re not going to run out of them @defunkt #RubyMidwest

Excellent point. How many gems have you deployed to production that claim to be version 0.194 or some such nonsense?

Define how to contribute to your project - don’t assume every knows how you want it. @defunkt #RubyMidwest
Be a lazy maintainer - Write a road map and people will do your work for you. @defunkt #RubyMidwest

Excellent point - this totally worked for metric_fu.

Link rot - don’t point to your svn repo if you’ve moved to git - update old blog posts if you use them as doc @defunkt #RubyMidwest
Google your project name before you create it - Find problems early @defunkt #RubyMidwest
Examples of great open source projects: Rails, JQuery, Redis, homebrew, Django, Unicorn, Linux, Git… @defunkt #RubyMidwest
Chris Wanstrath - @defunkt always delivers an great talk. He has a style that feels conversational but gets points across #RubyMidwest

Good talk - Chris is always an engaging speaker.

“Ruby Techniques by Example” Jeremy Evans is up next at #RubyMidwest
“Singleton class” has been said 20 times in the first 5 minutes of this talk @jeremyevans0 #rubymidwest

Yowza, kind of a rough start to the day. Had to stop tweeting for awhile just so I could focus (the horror).

ruby-sequel’s plugin method looks cool. Let’s you easily add modules that add class and instance methods @jeremyevans0 #rubymidwest
string evals dangerous but fast, define_method safer but slow. @jeremyevans0 #rubymidwest
The default argument of a ruby method can be any ruby expression. @jeremyevans0 #rubymidwest
Wrapping underling exceptions (from different implementations) with your own is a good to keep your code sane @jeremyevans0 #rubymidwest

Jeremy gave an intense, but good, talk. It’s had to make those deep internals of Ruby not sound dry but he did an admirable job.

“JRuby” by Charles Nutter is up next at #rubymidwest
Jruby: Real Threads and access to all of the Java. Compelling points. The slow test cycle (starting up the JVM) is a problem #rubymidwest
I know there’s solutions for starting up a test server, but I’ve found them wonky in the past - have they gotten better?

What I hear from my friends who develop in JRuby is that it’s awesome in prod but slow to develop in (you’re always waiting for the JVM to start up). And the lack of C extension support often means that the gem that would totally do what you need is just not going to work.

RT: @clam_tea #rubymidwest jruby allows you to do ruby on google app engine, android, anywhere java lives jruby is in the closet with a knife.
RT: @clam_tea Mmmmmm, Ruboto = Ruby on Android. Epic win. #rubymidwest
I like how @headius voice drops an octave when blatantly selling JRuby #rubymidwest
C extension support is being worked on by Google summer of code people for JRuby - Awesome. @headius #rubymidwest

If they can get C extensions working that would really be something. Maybe I’d have to give JRuby another look.

Next up is “Civic Hacking” by Luigi Montanez @LuigiMontanez at #rubymidwest
Sunlight Labs - developers and designers dedicated to improving government through transparency @LuigiMontanez #rubymidwest
Open Data + Open Source = Open Government @LuigiMontanez #rubymidwest
Of the hashtags #rmw #rbmw #rubymidwest, #rubymidwest is winning by volume For the Ruby Midwest Conference so that’s why I’m using it
FlyOnTime.us mines gov data to predict flight times. neat. @LuigiMontanez #rubymidwest
http://outsideindc.com/stumblesafely uses crime data to help you avoid crime when coming home drunk in DC @LuigiMontanez #rubymidwest
RT: @rumblestrut Thanks @LuigiMontanez, I somehow missed this entertaining drama that @aaronsw ran into last year http://bit.ly/1oZPmS #rubymidwest
http://codeforamerica.org/ Like Teach for America but with %90 less soul crushing @LuigiMontanez #rubymidwest
Service to your country doesn’t have to be joining government or the military - it could be code. @LuigiMontanez #rubymidwest

One of those awesome talks that points out some very cool things you could do with your open source time. Transparency is within our reach.

Free book time at #rubymidwest
Lunch was an enjoyable, if fairly standard, boxed lunch affair. Quite good quality considering the price of the Conf #rubymidwest
RT: @rumblestrut I love that @wycats was the first book winner at #rubymidwest book giveaway. Hilarious.
I just won a copy of “Beautiful Code” at #rubymidwest

Another book to read. At some point. w00t?

“Migrating Legacy Data” by Patrick Crowley & Rob Kaufman is up next at #rubymidwest
Trucker is a gem for migrating legacy data into a Rails app @mokolabs #rubymidwest
Trucker is hard to google: http://rubygems.org/gems/trucker @mokolabs #rubymidwest
“Trucker helps you move, but you still have to pack your stuff up” @mokolabs #rubymidwest

Trucker looks pretty cool and if I had some legacy to migrate, I’d look into it.

There are 12ish adapters for ActiveRecord, but there are tons of JDBC adapters. JRuby to the rescue. @mokolabs #rubymidwest
RT: @ajsharp “ALTER TABLE mytable CONVERT TO CHARACTER SET utf8” #rubymidwest

That last line is a particularly useful bit of MySQL that will help you convert to UTF8.

Next up is John Hwang (@tavon) with “Object Oriented Unobtrusive CSS” at #rubymidwest
Rename! the talk is now just “Unobtrusive CSS” @tavon #rubymidwest

He had to cut some scope to fit the deadline.

.@tavon says CSS frameworks (like 960grid) are almost as bad as using tables. Strong words. #rubymidwest

And he comes out swinging. I like it.

CSS is like assembly programming, frameworks - procedural coding, and Sass works at a higher level of abstraction. @tavon #rubymidwest
Sass: Nested Rules, Variables, mixins, compiles to css, Fully CSS3 compatible @tavon #rubymidwest
.@tavon says Sass is almost Turing complete #rubymidwest

These are great points. Sass lets you do things you should have been able to do years ago. How did we ever live without it? Wait, my project isn’t using Sass… I need to talk to my designer.

.@tavon feels like HAML is an unnecessary abstraction. Sass is a much bigger game changer #rubymidwest
“Programmers: HTML/CSS is your problem” @tavon #rubymidwest
“Programmers: You probably don’t know CSS” @tavon #rubymidwest
I think @tavon just called me lazy #rubymidwest

Hey now! I’ve been busy. I’ll get around to doing a deep dive on CSS. Really.

.@tavon will not work on a project if you won’t let him use Sass. #rubymidwest

A man of principles and conviction.

Stupid @tavon, making me feel bad for not knowing CSS very well #rubymidwest

He made my feel bad. Jake no like feel bad.

“Integration Testing Strategies: Locally And In The Cloud” is next at #rubymidwest by Ryan Felton and Kyle Ginavan
Live coding Capybara Selenium slowness and fail at #rubymidwest Sorry guys, I’ve been there
Using specjour to parallelize his capaybara tests on another machine. Go. Go. Go! #rubymidwest
And it worked. mostly. live demos are hard. #rubymidwest
http://www.seleniumshots.com/ is another an to run your integration tests in the cloud. Many have tried this. #ihopeitworks #rubymidwest
Don’t get me wrong, I want “selenium shots” to succeed but I’ve seen a lot of smart dudes fail at parallelizing in the cloud. #rubymidwest
RT: @bellmyer I just signed up for selenium shots! #rbmw #rubymidwest http://yfrog.com/1n9bcp
Re: Last re-tweet. Got a 500 after signing up for “Selenium Shots” #rubymidwest

There’s a lot of stiff competition out there trying to run tests in the cloud. May the best project win. Also, live coding and examples that rely on a network are to be avoided.

“Metawhat? A look into the mysterious metaclass” next by Brandon Dimcheff @bdimcheff at #rubymidwest

Another talk where I couldn’t tweet because I needed to focus. Intense and worth tracking down the video when it comes out.

“Redis Persistance Power” - Nick Quaranto (@qrush) happening now at #rubymidwest
Redis is Batman’s utility belt @qrush #rubymidwest
Redis can take a command that waits for something to arrive in the queue - meaning you don’t have to ‘sleep’ @qrush #rubymidwest
.@qrush is working on hosting Redis in the cloud http://akasentai.com/ Ready soonish. #rubymidwest

Cause that guy needs more side projects. And they’re all awesome. Yes, I’m jealous.

“Three thousand years ago, the Omnipotent Disc King started a campaign to ravage Sector 5 of the Moon.” from akasentai.com

Weird placeholder site: Go see it.

RT: @qrush My slides from Redis: Persistence POWER! (With way more examples than I could cover!) http://scr.bi/redispower

Bonus content? What a deal!

RT: @clam_tea $redis.sadd should have a $redis.happpy to balance it out.

Yes it should. Well not really, but fun to talk about but not do. Like yielding 42 for no reason in complicated bit of Ruby – funny but you wouldn’t want to actually do it.

RT: @pvisintin #RubyMidwest photos http://www.flickr.com/photos/peterpunk/sets/72157624517058396/
Side note: Kansas City and the University of Missouri-Kansas City campus are quite nice. #rubymidwest

It is a pretty town and university.

“Chef - Cooking 101” by John Williams (@j_m_williams) is up next at #rubymidwest
Chef: Automate building a server up from bare metal to a ready to deploy machine. pretty awesome. @j_m_williams #rubymidwest

I talked to John the night before and I think it was his first talk. He was a bit shaky. It didn’t help that server set up is dry, dry stuff. And, to be honest, most of the crowd doesn’t care about the topic. Sorry John but we’re coders.

One more Talk! “Surviving the last 20%” Mark Daggett (@heavysixer) at #rubymidwest
Speaking of Surviving the last 20%, I am mentally exhausted. #rubymidwest
I do! RT: @gregrhansen does anyone else see the irony of everyone leaving before the last talk of the day… the last 20%
“Facebook development is like building a ship in a bottle. Inside another bottle. While wearing mittens” @heavysixer #rubymidwest
RT: @tswicegood Regarding Facebook dev: FBML == Duplo, IFrame == Lego. #rubymidwest

Rough to be the last talk of the day. A good percentage of the crowd has bailed, lots of us would rather drive a spike through our cornea than write a Facebook app, and everyone in the room has a severe case of mental fatigue. Still he did a good job.

Demi Moore is to Facebooker as Patrick Swayze is to Rack. In the movie Ghost. In the clay spinning scene. @heavysixer #rubymidwest
You really kinda needed to see the slide to get that last tweet #sorry #rubymidwest

That was a great slide. Well played.

“Use Rack to wrap the plugins and gems to give them the params they expect instead of monkey patching them.” @heavysixer #rubymidwest

Clever solution. Mark strikes me a smart man fighting an insane API. And winning.

The talks are over for the day, back to the hotel, and then to http://omgwtfbbq.heroku.com/ Great name. #rubymidwest

It is a great name. From Urban Dictonary:

OMGWTFBBQ: A meaningless acronym which most often stands for “Oh My God What The Fuck Barbeque.” It most likely originated on Something Awful (somethingawful.com). It can be interpreted simply as gibberish, or used when one wants to emphasize one’s own incoherence, lack of understanding, or to mock others. It usually has an air of mockery, specifically with regard to teenagers who a lot of use three-letter acronyms.

Evans just gave a lightning talk on singleton classes of singleton classes and had the room rolling with laughter. Only at #rubymidwest

You sorta had to be there. The rest of the lightning talks were good, but I was too tired to Tweet it up so they will be lost to the internet forever. Apologies. The BBQ was excellent, however. I can still smell sauce on my hands and I’ve washed them 3 times.

Time for bed.

The Road to Ruby Midwest

Fri, 16 Jul 2010 03:47:00 +0000

Crazy day:
It was the day before Ruby Midwest so I gave my ‘Speedy Tests’ talk to my workmates at Backstop Solutions during lunch today and they had the nerve to find some things wrong with it – so I’ll be doing some re-tooling before Saturday. Then I was all set to leave 2 hours early for the airport when my manager wants to talk about me helping QA understand our process, new features, and maybe get some automated tests running. Hell to the yeah. So I’m all over this opportunity but our conversation makes me leave the office a bit late to catch my flight. No problem, I build in a lot of buffer. Then the Blue line (Chicago’s subway) was down. Uh oh. Some surly dude said there was a free shuttle to somewhere else but I couldn’t find it so I took a cab. Crazy traffic puts me at the airport with 30 min until my flight. Panic! But the security lines are longish so I panic while inching forward in a zig-zag line. Then I sprint to terminal C (which is the farthest away and involves going underground) and find my seat minutes before they close the doors.

The lady next to me wanted to make small talk about how I was out of breath. I did not.

Then when I landed I noticed that all the signs say “KCI” when I’m pretty sure I booked a flight to “MCI” so I had a minor heart attack moment where I thought I flew into the wrong city. But no, it turns out that KCI and MCI are the same. That’s not confusing at all.

But all was well when I made it too the speakers dinner. Very nice of the Ruby Midwest guys to organize such a dinner and pick up the tab. Good conversations with some locals (Shashank, Luke, and Josh), a 37 signals-er (John), and an Edge Case-r (Adam). Looks like the conference is off to a swimming start.

Rails Conf 2010 Day 3

Sun, 13 Jun 2010 18:30:00 +0000

A few days late on this because I’ve been sick all weekend. There just has to be a way to do climate control in conference centers in such a way as to not destroy the planet and, more importantly, Jake’s health. I seriously wore black jeans and 2 shirts on hot summer days and yet I was shivering and caught a cold.

But let’s get to last day of Rails Conf 2010 and my exciting adventures therein. (tweets are in italics)

Woke to “Relax” by Frankie Goes to Hollywood. Which is good ‘cause my heart was pounding. Settled it with some bacon. #railsconf
Yep, I could feel cold coming but I was in denial. I ran into Neal Ford at breakfast and we chit chatted about this and that and then he mentioned that he had read my blog post from yesterday. First thought: I published that sucker at 1am this morning and he’s read it already? Second thought: Oh shit, what did I say about his keynote – I think I called him a jerk at some point. But my brain was foggy and so I wasn’t quite sure what I had said. Later I looked and, yes, I had called him a jerk. I was kidding! If only that was the most embarrassing thing I did that day…

Pro tip: Have the hotel watch your bag for the day while your at the last day of the con #railsconf
It’s true. This year Rails Conf was nice enough to have a section for people’s bags at the conference but they don’t always.

Twenty-Five Zeros - Robert Martin

So, as many of you know, I started out at Object Mentor (as an unpaid apprentice/intern) and I’m a bit of an Uncle Bob fanboy. As these next tweets can attest:

Bring it Uncle Bob! #railsconf
Good to see Uncle Bob still starts all his talks with some hard core physics. #railsconf
HOLY SHIT!!! Uncle Bob opens by showcasing some impressive drum chops. #railsconf
RT @duncan: Photo of @unclebobmartin on drums at #RailsConf (ISO 12800 in the dark) http://www.flickr.com/photos/oreillyconf/4688547550/
@spiceee call him (@unclebobmartin) butter cause he’s on a roll #railsconf
#railsconf haiku Uncle Bob holds us We are enraptured by him Rambling? I think not!

Great speaker, although I had to de-follow him on twitter as I’m not into right wing politics.

If we have computers 10^25 times more powerful than 1960 then why no AI? Always seems 10 years out. #railsconf
Our software has not improved anywhere near 25 orders of magnitude unlike the processors @unclebobmartin #railsconf

Funny how social sciences never really advance as fast as the hard ones. I think of programming as a ‘soft’ science because the hard part is NOT getting a computer to understand the code it’s describing intent to humans.

.@unclebobmartin “What language do you know that still has goto?” guy in crowd: “PHP” #railsconf
A brief tour of programming languages with @unclebobmartin #railsconf
RT “vi? what the hell?” - @unclebobmartin #railsconf

Vi, Emacs, TextMate, etc. all seem like some weird time warp compared to the awesomeness of the language. It always seems like a really good Ruby IDE is just around the corner.

RT the point emerges. multicore processors pass the buck to us. future == parallelism #railsconf
Real innovation in software may be driven by the move to parallel processors @unclebobmartin #railsconf
.@unclebobmartin is hawking #sicp Yeah, yeah, yeah. I’m getting to it. #railsconf
#sicp is a page turner? Really? @unclebobmartin #railsconf

Structure and Interpretation of Computer Programs is so often praised that people in the know just refer to it as sicp. It’s available for free here: http://mitpress.mit.edu/sicp/ I’m thinking of working through it in clojure. And who doesn’t like LISP? Uh…

But his point about multi-core is important (although it has been made before). A lot of people took his talk to mean that Ruby will lead the way into the future of programming. I think his point was that the people at Rails Conf will lead the way – but probably not in Ruby. I’ve seen a few frameworks for handling concurrency in Ruby and they all suck. Why? Because you have to remember to use them all the time. All it takes is one programmer mutating a state somewhere in your code and now you have an intermittent race condition bug. If you’re really going to do concurrency you need a language that treats all mutation like a disease that must be contained in explicitly declared blocks. And that language is not Ruby.

But I think we still have a few years left before we all have to get functional, so that’s nice.

There’s 2 prominent black dudes in the Ruby community. I just gave a hardy hello to one and referred to him by the other’s name. #railsconf
Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. Sorry. #railsconf

In my defense: I’m an idiot.

You May Also Be Interested in: Implementing User Recommendations in Rails - Matthew Deiters

So Amazon makes 25% of its sales based on recommendations. I was talking with a friend from Groupon the night before and he was telling me that just guessing the sex from a person’s name (and targeting what they get offered) has led to a significant sales boost. If you’re not thinking about how to intelligently recommend to your customers, it’s a sure bet your competitors are.

Aaahh the Hoff’s crotch is coming at me! #railsconf
Oh it was just an animated gif. #railsconf

I urge you not to click this link: http://www.post-literate.com/gerpunx/archives/2005/01/prepare_to_lose_your_mind.php
I think Deiters was trying to make a point about recursion in SQL being bad, but the image made my brain stop working.

Gem Neo4jr-social is a graph db that uses JRuby but isolates it in a Jetty server so you don’t need to use JRuby throughout #railsconf
You can use Neo4jr-social to get friend recommendations and degrees of separation pretty easily. @mdeiters #railsconf
To which Charles Nutter (@headius) responded @jakescruggs Jeez, why not just use JRuby? So much pain could be avoided.
I really want @mdeiters slides for the explanation of all these crazy graph relationship terms #railsconf

Graph db’s are amazing for relationship stuff. You can do “what’s my degree of separation from X” stuff in milliseconds. And Neo4jr-social looks like a pretty cool way to do that stuff in a readable way.

You have a lot of money lying around in your data - pick it up! @mdeiters #railsconf
RT “This conference is the NoSQL conference. But remember: It’s not No SQL. It’s Not Only SQL” – @mdeiters #railsconf

Really good talk. Lots of info in the slides:
http://assets.en.oreilly.com/1/event/40/You%20May%20Also%20Be%20Interested%20in_%20Implementing%20User%20Recommendations%20in%20Rails%20Presentation.pdf

Lapidary: the Art of Gemcutting - Nick Quaranto

RubyGems.org has a nice versioned API @qrush #railsconf
‘gem yank’ to remove a gem. Didn’t know you could do that. There’s an undo too. Nice. @qrush #railsconf

Versioned API and an ‘undo’ – cool.

gem webhook projects suggestion - gem tarballer @qrush #railsconf
gem webhook projects suggestion - distributed testing service @qrush #railsconf
gem webhook projects suggestion - change log service @qrush #railsconf

All these ideas are great. I command the readers of this blog to go implement them now… Please?

Gemcutter moved from postgres to redis @qrush #railsconf mostly
Lots of Gemcutter clones out there @qrush #railsconf
.@qrush wants to add better indexes, dependency resolution, and support bundler in RubyGems.org #railsconf
.@qrush also wants historical data for every gem with download graphs. @qrush #railsconf
github.com/rubygems/rubygems No longer on svn. @qrush #railsconf

All very cool. Nick is clearly firing on all cylinders – he deserves a raise.

Yehuda Katz - showed up and thanked Nick for his contributions to the Ruby community. I was in quite a few talks where he made it a point to thank the speaker for things he really liked. Stand up guy that Yehuda.

RT: OH: “Firefox is the new IE.” #railsconf (via @glv)
Ouch. But kinda true.

The Cold/Hot/Cold/Hot/Cold/Hot/Cold/Hot of this week has made me weak. Picked up some knockoff airborne at the CVS #railsconf
Getting sicker… Must consume more vitamins and wear more clothes. Covering up crazy shirt now. :( #railsconf
Must. Fight. Sickness. At. The. Airport.

At that point the sickness overtook me so I found a quiet place and laid down. So I missed the last keynote – which I’m about to watch now.

And I’m back. Good Keynote. Actually, all the keynotes were pretty darn good this year. And so were the sessions. Shortly before I got sick on Thursday I remember thinking “This is probably my favorite Rails Conf.”

Gary Vaynerchuk’s talk is probably summed up best with this (made up) title:

Relationships, relationships, relationships, (and swearing) - Gary Vaynerchuk

Here’s some memorable quotes and thoughts:

“Giving a Fuck is coming on strong”

“Stop using the space (twitter, facebook, etc) just to put out fires”

“If I get a hundred more followers I’ll donate $100 to Haiti - Hey Fuck-face just donate $100 to Haiti”

“Everything you’re doing is being documented”

“It’s just getting real hard to hide”

In 2005-6 everything was free 24/7/365. Now people are being trained to pay for things.

On consulting for huge companies: Its stunning how little most big companies give a crap. Most CEO’s want to keep the stock price up for 3-5 years and get out with a huge payday.

Getting people to your site is awesome but “Content is always king”

“What’s the ROI (Return On Investment) in social media? Well I don’t know fuck-face what’s the ROI in having a real relationship? Meanwhile you’re paying for billboards…”

Old businesses: “They lived under small town rules” - if you screw someone you’re going out of business. Those days are coming back.

Corporations: “We don’t want to open this up because people could say our product’s bad”

Gary on the future: “I’m all in – I’m bullish on human beings.”

You can find Gary’s Keynote along with all the others at: http://www.youtube.com/view_play_list?p=393ECE649BB3813D or on iTunes.

Well that’s the end of my Rails Conf 2010 coverage. Thanks for reading.

Rails Conf 2010 Day 2

Thu, 10 Jun 2010 04:35:00 +0000

Only 40 tweets today (re and otherwise). I must be slowing down in my old age. Tweets are in italics.

Today’s morning music was “Bulletproof” by Pop Will Eat Itself. Cram that in your head @ryanbriones
Ryan was complaining yesterday that he had the theme from flash stuck in his head all day because of me. That sounds awesome! I had a Daft Punk song stuck in my head all day today and I was a better person for it.

Trying to work on my Lightning Talk. Lightning talks are usually the best part of any conf so it better be good. 4:25 today #railsconf
Lightning talk is finished. Title: “ActiveMQ and ActiveMessaging: I’ve Experienced the Pain So You Don’t Have To” #railsconf

Yeah, I got up, worked out, ate eggs wrapped in cheese and bacon, showered and then realized I had gotten up an hour early by mistake. Opportunity! So I worked on the lightning talk. To be honest, I was kinda pissed at myself for the boneheaded alarm mishandling but if it doesn’t make you money or happy, you need to let it go.

Don’t forget to read my sham of a blog post about #RailsConf Day 1: http://is.gd/cIdLZ
So very meta.

iPod at #RailsConf Keynote has been set to “BadPopCountryMusic” wtf mate?
Seriously. Is there anything worse than top 40 wanna be country music?

Creativity & Constraint - Neal Ford

“Some of the ‘facts’ in this talk may in fact be harmless lies” - disclaimer of @neal4d #railsconf
This disclaimer needs to be in front of all keynotes.

I’m Not Old, I’m seasoned. @neal4d #railsconf
Me too buddy, me too.

“The right community to suggest something really weird to” @neal4d #railsconf
RT I so wanna go to Neal Ford’s halloween party! #railsconf
Constraints are liberating: @neal4d excuse for showing cool Halloween photos at #railsconf

His Halloween party pics were off the hook. Did I ever tell you I took Neal to his first day lunch at ThoughtWorks? You’d think that would get me a Halloween party invite but memories are short. Jerk.

What constraints help? Maybe not static typing… @neal4d #railsconf
Double true.

Hemingway and the Ex Pats in Paris wore stupid hats #railsconf
Well they do. Americans in berets look stupid. There – I said it.

An empty canvas is a daunting task @neal4d #railsconf
The invention of the camera made painters do something different besides making it look real. @neal4d #railsconf
The Camera gave painters freedom from the responsibility of representation. @neal4d #railsconf
Our camera is Java Enterprise Development - they can crank out vanilla apps so we can move on to interpretation @neal4d #railsconf

Interesting. I think he meant no offense to Java devs in specific but much offense to the mindless and complacent.

.@neal4d has a crazy terse 3 line Quicksort implementation #railsconf

Yeah, I had no idea what it did. You’d need a team powerful enough to turn goat piss into moonshine if you’re going to check that in.

Non-obfuscating density is a characteristic of art in code @neal4d #railsconf

But how much density is too much? It’s a thin line between art and crap.

RT #railsconf @_why : when u dont create things, u become defined by ur tastes rather than ability. ur tastes narrow & exclude ppl. so create

Neal ended up his talk with poignant quote from _why. Pour one out for the homies who couldn’t be here.

Sponsored Keynote: Engine Yard’s Open Source Love Affair - Evan Phoenix

Dr. Who Theme! Played live by #railsconf ers
Huh?

Did Evan come on as a rock star or a parody of a rock star? you decide. #railsconf
Rim shots at a keynote? #railsconf
Talk show format is… interesting Will it pay off? @evanphx #railsconf

A ‘Late Night with X’ theme backed up by a live band. And a game of password.

.@evanphx “Perl is the word you were looking for” Dr. Nic: “Never heard of it.” #railsconf

Well played Doc.

Enjoyed a pleasant 10 minutes of nothing with @evanphx #railsconf

I really did enjoy it. Better than somebody coming out and saying “hey we bought this slot with our sponsorship and we’re awesome.”

And then Twitter went down. Again. So I had to type into a document instead of live to the world. It felt weird and dirty.

Ruby on Rails: Tasty Burgers – Aaron Patterson

First, you have to download his slides just for the introduction: http://tenderlovemaking.com/railsconf2010.pdf

Two samples to entice you:

Btw his outfits put mine to shame. Today’s was a deep magenta velvet jacket with a 70’s tie and an ironic mustache. Or a sincere but poorly chosen mustache. There’s a point when intense silliness becomes serious and upon that point Mr. Tender Love Making is doing an interpretive dance.

Topics he talked about:

sqlite3-ruby and Sqlite

Did you know you can have a virtual file system in Sqlite? Me neither.

The new version of sqlite3-ruby is x1000 faster than before (1.3.0 vs 1.2.5)

Mmap

Mmap is a gem that lets you treat a file like a string. So if you need to pass in a string to something but don’t want to read the whole file into memory use Mmap. Pretty useful tool to have in your back pocket.

Rack

Unknown but useful rack trick: Rack will call ‘each’ and ‘close’ on the return object. So you can pass it an array or file and things will work out just fine. Handy.

He speculated that Rails could take advantage of this and return html in pieces (as it’s rendered) to make things faster. Would be hard to do but awesome.

JSON is a subset of YAML – I did not know that. Did everyone else? Really? I knew you all were talking behind my back.

He left the stage with a plea for the audience to dig into rails and investigate the ‘Tasty Burgers.’

But wait, there was some advanced shenanigans when he called Evan Phoenix to ask him a question. During Evan’s talk! Those guys are the kings of wacky.

Redis, Rails, and Resque - Background Job Bliss by Chris Wanstrath

No Slides at all! But still good.

“Redis is all we really wanted it for a long time but we didn’t know we wanted it”

Redis is a key value store for data structures @defunkt #railsconf

This doesn’t have anything to do with Chris Wanstrath’s appearance but comical mustaches are the new black. Except that he has a mustache now. And it looks silly. Not that I can complain about peoples fashions sense.

Chris is totally in love with Redis and Unicorn because their maintainers are awesome, active, and their software is pretty bullet proof (it stays up). Which is kinda important to Github.

He advises you to background everything you can and I agree.

Resque is the queuing system they built on top of Redis. I’ve used it at Groupon and seen it used on Mad Mimi and it works well. Plus the built in Sinatra web UI provides an amazingly informative look into the live queues.

Github is on Ruby Enterprise Edition now. So is Backstop (my company). And everyone else at Ruby Conf. It just surpassed ironic mustaches as the new black.

Chris is determined to not add any queuing features to Resque. He wants to let the plugins do that and there are more than a few you should check out. I’d Google that for you but I’m tired.

Rocket Fueled Cucumbers - Joseph Wilk

So Joseph’s cuke build was getting out of control as it grew. Taking more than 4 hours. He solved this by using EC2 and Testjour (or Hydra) to parallelize the suite across 20 machines and got it down to 11 minutes. But it cost $3K a month. Yipes.

He recommends stubbing out slow services and maybe even caching the db (but didn’t specify how).

He’s excited about using the cucover gem to do “Lazy coverage-aware running of Cucumber acceptance tests” Kinda like a super smart autotest.

He mentioned this post as required reading that he doesn’t necessarily agree with: http://jamesshore.com/Blog/Alternatives-to-Acceptance-Testing.html

Capybara can use envjs gem to run javascript tests in cuke. Which is very cool. Capybara-envjs on github

Dear Lazyweb, how do you get a showoff presentation up on Heroku? I’d like to put my lightning talk up. #railsconf
Posted this before my talk. Still have no answer but if all the cool kids can do it I’m sure I can figure it out.

Lightning Talks

Lightning Talks start now and I’m giving one. Come to Ballroom 1 now! #railsconf
Yep.

RT client_side_validations http://bit.ly/9gEfuL #railsconf
Interesting gem. Way too much live coding for a lightning talk - dude was brave. He barely made it to the end and almost missed giving out the gem name. Live coding seems like a good idea but is to be avoided in presentations. Ignore my advice at your own peril.

Don’t hide costs from the client – it leads to bad relationships #lightningtalks #railsconf
Custom filters based on metadata looks cool in RSpec 2 #lightningtalks #railsconf
gem install surveyor for easy surveys in your rails app #lightningtalks #railsconf
surveyor gem can ask some complicated questions with easy. #lightningtalks #railsconf
RT Mind. Blown. http://bit.ly/90D1ur Now that was a lightning talk by @igrigorik. #railsconf

Yeah the schema free sql database thing was a nice antidote to all this NoSql hype.

http://bit.ly/RwandaOnRails looks like an interesting way to track charity money’s effectiveness #lightningtalks #railsconf
“Yes and…” is an important part of improv and pairing. I totally agree. #lightningtalks #railsconf

My lightning talk went well and as soon as I figure out how to get it up on Heroku I’ll give you (dear reader) the url.

Update: Here they are: http://active-messaging-pain.heroku.com/

Keynote - Derek Sivers

In 2007 Derek took a lot of heat for his article entitled: “7 reasons I switched back to PHP after 2 years on Rails”
Which can be found here: http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_switched_back_to_p_1.html

I remember reading it and thinking he was a moron. He gives a pretty good keynote for a moron so perhaps I should reevaluate.

RT The first follower turns a lone nut into a leader. - @sivers #railsconf
RT derek sivers shows this vid http://bit.ly/guHuz to talk about being a leader and starting a movement #railsconf

The talk was all about how he preferred his own PHP frameworky thing but how that couldn’t scale and so he’s coming back to Rails. And is excited about Rails 3. Also there was a lot about building community. We use our emotional brain to justify our rational experiences. But sometimes our emotional brain has access to things we don’t consciously know (detailed in the book ‘Blink’). There was more but it’s late and I’m pretty tired so I’m wrapping this blog post up with a few more tweets:

Fogo de Chao ftw #railsconf
Drunk. Drunk on steak. Drunk on chocolate. In fact, I’m like a chocoholic except for alcohol.

Rails Conf 2010 Day 1

Wed, 09 Jun 2010 03:57:00 +0000

I tweeted 73 times today. What the hell is wrong with me? When twitter went down in the middle of the day I panicked until I realized that things continue to exist even if you don’t tweet about them.

Anyway, here’s my thoughts on Rails Conf Day the First interspersed with relevant tweets in italics and director’s commentary.
(Is this Lazy? Yes – Bite me.)

Woke up to “Theme Song from Shaft” – Bring it #railsconf
Set the old iPod classic to some inspirational music to psyche myself up for a day of interacting with strangers. 160 Gigs of storage – take that solid staters.

I ate my customary breakfast of bacon topped with eggs, salt, and cheese and then was off to the castle-like mammoth Baltimore conference center.

Keynote - David Heinemeier Hansson

Number 1 Fave of DHH in Rails 3: Bundler. #railsconf
Nice to see some validation of my blatant fan-boy cheering on of Bundler. DHH loves it so suck it, haters.

DHH - “When I would swear Bundler was a piece of shit it usually turned out to be me not listing a Gem” #railsconf
Yep, just when I think it’s Bundler doing something wrong, it’s not. It’s the canary in the coal mine telling you about gem problems in your app. As such it gets blamed a lot for problems out of its control.

DHH - “Hashes are great but we’ve kinda been drunk on them for a long time” #railsconf
The new ActiveRelation gets rid of hashes in favor of method calls (.order vs :order => …) #railsconf
DHH - ActiveRelation makes building up queries much easier. #railsconf

The ActiveRelation stuff (covered in yesterday’s post a bit) does a good job of fighting ‘hasheritous’ and making queries readable. No small feat.

If you just please your core users you create an impenetrable framework. Stepping away from that is what makes Rails 3 great. #railsconf
This may be the most important but overlooked thing about Rails 3. Keeping things easy to get into and read while increasing the power of Rails is a powerful combination that will increase adoption.

DHH is bashing on the old routes format: concise but unreadable. #railsconf
DHH - “there’s just too much hashing going on” in routes #railsconf
DHH - The majority case is now complicated routes files and Rails 3 is trying to make them more readable. #railsconf
map.with_options is now a scope – looks cool #railsconf

This is something like the 4th re-write of routes. But the emphasis this time is on readability. Which is awesome as almost all routes files were becoming impenetable piles of hashes that did some bad ass shit. Rails 3 increases the bad ass but adds some intentionality.

If you like the new stuff in ActiveRecord, thank @miloops, whose GSOC project integrated Arel. Great job Emilio! #railsconf
True dat. Mad props to Emilio.

@dkastner Why not use Ruby for configuration – I understand it better than the pitfalls of yaml
I was responding to this: “Hm, so the question is, why are we using Ruby to define our routes? Do we really need loops and conditionals?” Basically I love config in Ruby for reasons stated above.

DHH’s 4 faves of Rails 3: Bundler, Active Record Queries, Router, and ActionMailer #railsconf
The official @railsconf twitter account retweeted this tweet. I’ve really arrived.

DHH - “Mailers for a long time had a split personality” Half controller and model and the worst of both. Controller won in R3 #railsconf
Complaints that I and others have been making for years. Glad to see Mailers finally becoming one thing.

Signed permanent cookies in Rails 3? Neat. #railsconf
I had not heard about that.

DHH - wants asset pipeline for Rails 3.1 #railsconf
May I suggest http://github.com/aberant/css-spriter for creating sprites? as part of asset pipe-lining #railsconf

Would be cool to handle css, javascript, and images not as a bunch of junk in the public dir but as first class important things. Also, why not use css-spriter for creating sprites of your many images? Full disclosure: Css-spriter was written by some friends of mine. But they’re real smart so you (and Rails) should use it.

Keynote - Michael Feathers

Hard - breaking dependencies to write usable tests in java/c# - Not so hard in Ruby @mfeathers #railsconf
Novices start out writing good code, but as they become experts they write complex code as they understand too much @mfeathers #railsconf
Big Ball of Mud from @mfeathers talk http://www.laputan.org/mud/ #railsconf
“Your commit history is a Gold Mine” Most change areas are important to know about. @mfeathers #railsconf
“Find the most executed path and hammer it” make it awesome and tested @mfeathers #railsconf

Hard to follow DHH’s Rails 3 is ponies and rainbows talk with a thoughtful one but @mfeathers did a good job #railsconf

Michael’s talk was good stuff but it did come after the excitement for DHH’s keynote so I hope people could shift gears and appreciate his insights. He’s not always the most dynamic speaker but “Working Effectively with Legacy Code” was an important book in my development as a coder so he definitely has good ideas.

Real Software Engineering - Glenn Vanderburg

Winston W. Royce wrote a paper entitled “Managing the development of large software systems” which is generally credited with starting waterfall. However it was actually designed to be a warning against waterfall software processes.

People who designed software engineering misunderstood 2 things: Software and Engineering @glv #railsconf
Royce: defined waterfall and said it would not work. The world saw the definition is easy terms and adopted waterfall. @glv #railsconf
Royce: An implementation plan keyed only to these steps (waterfall) is doomed to failure. @glv #railsconf
Of all the thing’s in Royce’s paper (that started waterfall) the easiest to understand was waterfall so it was adopted @glv #railsconf
There is no kind of engineering where you put something in one end and turn the crank. @glv #railsconf

I signed up for a Lightning Talk tomorrow at #railsconf I wonder what I’ll talk about…
I should probably figure that out. Soon.

Build models and test them. Early bridge makers new this before they had the math to prove things. @glv #railsconf
In fact when math came along, it was incomplete and led to the Tacoma Narrows bridge collapse - testing would have helped @glv #railsconf
Mathematically modeling was introduced (to engineering) as a cost saving measure @glv #railsconf
The source code is the design of our system @glv #railsconf
Source code is math that does real work. @glv #railsconf
RT “SW engineering is the science & art of designing & making… systems that can adapt to situations they may be subjected to” @glv #railsconf
Agile processes are optimized feedback engines @glv #railsconf
RT Glenn Vanderberg is the Garrison Kellior of software engineering conferences. #railsconf

Good thoughtful stuff. Glenn later said to me that he’s dying to give this talk somewhere where he isn’t “preaching to the choir.” Check out his slides for cool quotes at: http://assets.en.oreilly.com/1/event/40/Real%20Software%20Engineering%20Presentation.pdf

Lots of “Fail Whale” today. Twitter can handle Ashton Kutcher but not #railsconf
The dark time of the day when twitter went down. The horror, etc.

Metrics Magic - Aaron Bedra

Seems like people really like failing the build on bad metrics – I should really get around to putting that into metric_fu #railsconf
Churn mentioned in @abedra talk can be found here: http://rubygems.org/gems/churn #railsconf

Churn is a gem extracted from metric_fu that looks at your source control to see which files change a lot (possibly too much – god objects).

Liked @abedra talk on metircs - he focuses a lot on failing the build with metrics #railsconf

Check out Flack-jacket if you want to fail the build on poor metrics results: http://github.com/abedra/flack-jacket

Saw a “Save the princess, Save the world shirt” with a image of Link. Nice. #railsconf
Old Heroes ref (remember when that show was exciting? Me neither – its been too long.) but still funny.

@dastels Totally – I got a free pony plushy for signing up with http://workbeast.com/ Haven’t seen any rainbow swag. #railsconf
Dave Astels was giving me (and the rails community) crap about Rails 3 being ponies and rainbows.

Garbage Collection and the Ruby Heap - Joe Damato

GC talk is cool. We really need to tweak our GC settings on REE to get better performance. #railsconf
Where are you leaking memory? Memprof knows. It rewrites your Ruby binary in memory to do deep awesome. #railsconf
memprof.com “a web-based heap visualizer and leak analyzer” Sounds good but site is showing a passenger error page now. #railsconf
@aslak_hellesoy Download the PDF @ http://timetobleed.com/ Looks better on your laptop

Joe’s slides had poor contrast - luckily they were available online. Check ‘um out, there’s some good info there.

Garbage Collection and the Ruby Heap with Joe Damato was technical but still graspable. Slides here: http://timetobleed.com/ #railsconf
I’m old and repeat myself sometimes.

http://memprof.com should totally get bought out by @newrelic and the coolest performance site eva! #railsconf
New Relic totally owes me a %10 finders fee for this idea. It would be teh awesome.

Later http://memprof.com came back up and I looked around. Nice interface and cool site.

Ruby’s Dark and Dusty Corners - Evan Phoenix, Charles Nutter

I loved Charles’ 3 rules for threads:
1. Don’t use threads
2. If you do, don’t share state
3. If you share state, don’t share mutable state

word.

Interesting super nerdy language implementation talk here in the “Dusty Corners” talk. Some important caveats. #railsconf
RT #railsconf : just say no to _id2ref!
“Memoized values are a memory leak.” #railsconf
GC.start - MRI: sometimes helps, REE: better tuned not needed as much, JRuby, Rubinius, MAcRuby, IronRuby: May actually hurt #railsconf
More sophisticated GC implementations need more ‘room to breath’ but are faster. #railsconf
About GC: Memory is cheap, CPU cycles are not. #railsconf

GC is important, you should probably take some time to look into it if you need to make your app perform better.

Keynote - Yehuda Katz

Rails: Make hard things easy and impossible things possible #railsconf
Slogan stolen from Perl and modified.

“Find something that looks impossible and then do that” @wycats #railsconf
Both @wycats and I started programming in earnest in 2004 #railsconf
We both also used Front Page in the early days. It’s like we’re twins. @wycats #railsconf
Even if you are a total noob you can help @wycats #railsconf
Rewriting TMail and ActionMailer seemed impossible but it’s done and in Rails 3 @wycats #railsconf
Lots of dudes without CS degrees and no followers on twitter so they were not in the ‘in’ crowd. But they are now. @wycats #railsconf
People say Ruby will never be as fast as C. Who wants to bet against the JRuby and Rubinius boys? Not me. @wycats #railsconf
The Secret to getting into the ‘in’ crowd: Almost nobody turns down an offer to do hard work. @wycats #railsconf

A really inspiring speech by Yehuda. I had no idea that he, like me, didn’t have a CS background.

Ruby Heroes Awards Ceremony - Gregg Pollack

Congrats @qrush on your Ruby Hero award. Well deserved. #railsconf
Congrats on your Ruby Hero Award @wayneeseguin How could I live without RVM? It wouldn’t be worth it. #railsconf
RT 2010 Ruby Heroes: @josevalim, @qrush, @fxn, @tenderlove, @wayneeseguin, @seacreature – well deserved, great job! #railsconf
Take the initiative this year. Don’t ask permission. Think it, write it, ship it. @greggpollack #railsconf

The Ruby Hero Awards are very cool. Kinda made me feel like a slacker but that’s OK. Do as much as you can and the rest will sort itself out is my motto.

Long day of shaking hands and kissing babies for an introvert. Time to hide in my room and call the wife. See ya tomorrow #railsconf
Yep.

@polgardy Perl was credited in the talk, but the slogan was improved upon.
Fred was asking about the Perl slogan mod.

@dastels GC has, sadly, not been ‘solved’ in Ruby. It’s finally getting the attention it deserves.
@dastels Why so long to take GC seriously in Ruby? A. You solve problems when you need to. B. The East/West Ruby/Rails divide.

Dave wanted to know why all the tweets about Garbage Collection.

Time for bed. Thanks for reading this sham of a blog post.

Rails Conf 2010 Tutorial Day

Tue, 08 Jun 2010 00:41:00 +0000

Here I am at Rails Conf 2010 in scenic Baltimore, MD and these are my thoughts on the tutorial day.

The Rails 3 Ropes Course - Gregg Pollack and his band of minions (Nathaniel Bibler, Thomas Meeks, Jacob Swanner, Tyler Hunt, Mark Kendall, and Caike Souza)
The worst thing about this presentation is that is was so informative, slick, and well paced that it spoiled me for the afternoon session – But more on that later. Gregg et al. took on an ambitious task and knocked it out of the park. They discussed a number of new things in Rails 3 and had 5 labs to work through. Every time I was just about done with my lab time was just about up and we moved on. Speaking as a former teacher, that is damn hard to do.

Details:
When setting up a Rails 3 app you can do some interesting things like skip active record, or test unit, or git (What? By default Rails 3 will create an intelligent .gitignore file for you as every Rails developer uses Git (heh)). You can do things like this because of the increased modularity of Rails 3.

The new routes syntax is fun:

match ‘login’ => ‘session#new’, :as => ‘login’, :via => :get
get ‘/articles/(:year)’ => ‘articles#show’ # The :year is an optional parameter as it is inside parens.

You can add constrains like so:
:constraints => {:user_agent => /iphone/}

In Rails 3, Bundler is used to manage gems. I’ve used bundler in a few production apps now and I’m totally in love with it. Bundler does what config.gem was supposed to in an easy and awesome way. And you can use it in 2.3.5! Bundler Rulez! OK, I’ll stop now.

ActionMailer now extends from AbstractController.

O.
M.
G.

Now mailers aren’t this weird other world that’s kinda like a model but kinda like a controller. It uses the same underlying code as the controllers. This is a long needed refractor that has enabled better syntax and less confusion.

ActiveRelation is the new bad-ass relational algebra querying syntax where you can do cool stuff like:
Post.where(:author => “Joe”).include(:comments).order(:title).limit(10)

To sum up ActiveModel, all of these things that used to be tightly coupled with ActiveRecord:

Callbacks (before and after hooks)
Dirty
Errors
Naming
Observing
Serialization
Translation
Validations

Are not anymore and you can use them in ordinal Ruby objects just by including a mixin. Sweet.

Rails 3 assumes any string is an XSS (Cross Site Scripting) attack unless you tell it otherwise and escapes it. To output html in a view, you’ll need to use the ‘raw’ method.

Unobtrusive JS is now used in all the view helpers so if you’ve ever felt bad about using :confirm => “Are you sure?” but did it anyway because it was just so easy, you can put down that bit of guilt as it all reference a proper Javascript file and with no inline JS.

Rails 3 truly is ponies and rainbows.

Rails 3 Deep Dive - Jeremy McAnally

Tweet from Jeremy McAnally (@jm) 2 days before Rails Conf: “Why wouldn’t I catch the flu right before RailsConf where I’m supposed to talk in front of people for a few hours? *sigh* Vitamin-C, engage!”

So the first 10 minutes were about how Rails 3 is basically a Rack app. And then, all of a sudden, we were in a lab. He gave us 3 possible things to build with either Rack or Sinatra. With no explanation of how to do so… So I started to google for Rack docs, but 2 thoughts occurred to me:
1. I could google on my own time.
2. This does not bode well for the rest of the talk.

So I bailed. It could have gotten awesome after that, but the tweets were not kind.

I ended up at:

Contributing to OSS - The 8 Steps for Fixing Other People’s Code - Dr Nic Williams

Which was… Well right after I showed up he had a ‘lab’ where you were supposed to meet people until you ran into a foreigner. In a way it was valid to have us meet people as open source is all about collaboration. In another, much more real, way it seemed like he was trying to pad a short presentation. He encouraged us to find an open source project an write a ‘how to contribute’ section for it because almost none of them have one. I then realized that metric_fu doesn’t have such a section and wrote one. After that there was some floundering around with git and everyone trying to use the wireless at the same time. I wandered away.

And into an awesome discussion at BOHconf (an open spaces, or ‘un-conference’ that is running concurrently with Rails Conf at the Baltimore conference center) about fast tests. I got way more out of that half hour than the previous 3 hours combined.

Things I’m planning on looking into:

http://grease-your-suite.heroku.com/
Turning off your atime and journal data write back for faster file IO (don’t try this in prod).
Looking at hard disk vs cpu activity during tests.
Tweak your test database to delay time to write back and increase its cache size
Using Hydra to parallelize your suite – http://rubygems.org/gems/hydra
Specjour - RSpec with bonjour

Things that came up which I hardily recommend:

Use Ruby Enterprise Edition and set your GC settings to be absurdly wasteful.
Get rid of fat factories (Code that generates a model with lots of models hanging off it – all of which need to be created in the db).
When using Factory Girl (or any other factory that generates ActiveRecord models) just create an instance instead of always saving it to the db. Most times you don’t need to touch the database.

Somewhere in there Nick Gauthier declared: “I love making test suites faster. Tell anyone who wants to come by BOHconf and I will make their test suite faster.” Take him up on it is my advice.

And then it was time for steak and rum.

Interview Coding Problems

Tue, 18 May 2010 23:06:00 +0000

One of the awesome things about the business we’re in is that you can ask people to actually do the thing you’re hiring them to do IN THE INTERVIEW! You can’t really ask a banker to do some banking during an interview, but it’s relatively easy to set up a computer with a coding problem and ask a potential hire to work through it. Having recently gone through a job search, I though I’d discuss my thoughts on coding problems as part of the interview process.

The first time I ever wrote code for an interview was at ThoughtWorks. It was a 3 day at home coding problem. There were always 3 different questions and each could be knocked out in a few hours. The candidate could choose which one they wanted to solve and upon submission, the solution was looked at by at least 2 different developers. The great part about this is that we could filter out a lot of candidates before bringing them in to the intensive day long interview. The downside of a take home test is that you never really know if they got some help. Which is why in this type of interview it’s important to go over their solution in person.

One coding interview I did was 2 developers and I working on a Rails plugin. It seemed like a good idea, but I don’t create Rails sites from scratch every day so there was a fair bit of fumbling around hooking up the database and trying to remember what generators to use and so on. When we finally got to the writing a plugin part I was a bit flustered and short on time. We made some progress but I don’t think either of us were satisfied with the outcome. This brings me to an important point: You will accomplish a lot less in an hour than you think you might. This is because the interviewee is using an unfamiliar machine, text editor, doesn’t have his or her custom bash aliases available, and is being watched. And questioned. All this will slow anyone down by 50%. So if it would take you 30 minutes to solve a given problem from scratch then it’s about right for an hour long pairing interview.

At Obtiva, Colin and I paired on some truly horrible code that he had inherited. Oh the stories we could tell you about that code base… But I’m not sure how I feel about these “work with me” interviews if they are going to last less than 3-4 hours. Actually, a day would be a better time allotment for this type of interview. The reason is that solving a problem in an existing code base requires a lot of backstory. If you only have an hour or two it’s hard to grok the domain and you may just be testing the candidate for familiarity with the subject matter instead of pure programming talent. It would be easy to miss a good candidate or hire someone who just happens to know the domain.

Speaking of domain, you should try as hard as possibly to eliminate it from your coding problem. I once did an interview with a firm where the challenge was to build a highly simplified version of something from their domain. The problem was that it really wasn’t simplified enough. After reading a page and a half describing the system my head was spinning with new terms and concepts (I wasn’t familiar at all with their field). They had helpfully started me out with some stub classes and some failing cucumber specs but when piled onto a shaky foundational understanding it actually hurt more than it helped. I had to figure out the domain, understand the class stubs (and any hints they may have been providing), describe what I was doing to the interviewer, understand what the cucumber tests were asking for, and navigate an unfamiliar workstation. It turned into a bit of a train wreck. Basically I did what I do when I don’t understand something: I went slow and tested everything. But I ran out of time before I could even get one of the cucumber scenarios passing.

You should try to structure the problem so that it has an easily achievable first step. This enables the interviewee to get his or her confidence up to try the trickier parts and also provides an easy hint you can give them if the are struggling. You might say: “Hey, why don’t you just try and get X working.” X being the easy part. You’ll get better results when people are relaxed.

At Backstop we have a couple of coding questions, but the central theme is that the domain is either something everyone is totally familiar with or we can teach them in a few minutes before the coding starts. We also make sure that the candidate pairs with different programmers to see how they interact with different people. Now pairing while interviewing is not like regular pairing, of course. When I run the interview, I approach pairing with an interviewee like I’m a novice developer who is a domain expert. I’ll answer any questions they have about the domain but I don’t help much with the coding part. I always allow access to any documentation they want to look up and of course I have irb open and ready for experimentation.

For me the pairing interview is just a way to have an engaging conversation about code. If someone doesn’t solve the problem but does so in an interesting and thoughtful way then I consider it a success. This also means that an unreadable working solution is considered a failure.

Having actual programing be a part of your interview process will take a some thought, but all that time is well worth the effort. I can’t tell you how many times someone has sailed through the talking parts of the process only to crash and burn in the coding section. Hiring someone who is great at talking about code but terrible at actually coding is one of the worst things you can do.

Upcoming Speaking Dates

Tue, 18 May 2010 01:45:00 +0000

I’ll be giving an all new talk entitled “The Necessity and Implementation of Speedy Tests” at Ruby Kaigi in Japan! Ruby Kaigi is being held August 27-29th, 2010 in Tsukuba, Ibaraki, Japan. Very exciting!

Want to see the above talk but can’t make it to Japan? I’ll be giving the same (ish) talk at Ruby Midwest in Kansas City, MO. Ruby Midwest will be held July 16-17, 2010.

Stop on by and say hello – I’ll be the guy in the crazy shirt.

Why I Left Obtiva

Mon, 17 May 2010 01:20:00 +0000

Recently I ended my nearly 2 year relationship with Obtiva and why I left deserves a few words.

The issues I had with Obtiva were mostly just issues I have with consulting:

You only work at places messed up enough that they need to hire consultants
You leave before you get to spend time with code you wrote so it’s hard to learn long term lessons
The most stable gigs that pay the bills for a consultancy tend to be the worst programming experiences (Large corporations with way more money than sense)
You have to track every hour of your time
Time off has to be negotiated with two companies instead of one
Learning a whole new bunch of names. personalities, and organizational structures every 6 months

Now don’t get me wrong, consulting with ThoughtWorks and then Obtiva made me a much better programmer and was an amazing learning experience but I felt that, in order to grow as a programmer, I needed to work for a product company and learn the lessons that only living with a code base long term can teach you. Consulting is a great drinking-from-a-fire-hose experience that every developer should consider, but it has its limitations. So I joined Backstop Solutions and I write Ruby on Rails sites for hedge funds and the people who use them (note: we sell products to hedge funds, I don’t actually work for a hedge fund).

Why Backstop? Well, it’s big enough (60ish people) that they’ve worked out most of the kinks of being a business (direct deposit, 401k, health insurance, etc.) but small enough that you can get to know everyone and (hopefully) help steer the company. Their product is strong and growing, but there’s some new products in the works too. They do Agile but would like to do it better (I’ve been brought on to help with that in addition to my coding responsibilities). But the best part was that while everyone is smart and nice, it’s oddly unknown in the Ruby community – and I can help with that.

Obtiva was a great experience. During my time there I moved from being just a coder to being more of a leader, started speaking at conferences, got to participate in many wonderful geekfests, helped mentor software apprentices (which I once was), and learned a ton from all the programmers I interacted with.

To sum up I’ll quote from my company-wide resignation email: “Mommy and daddy still love each other. Except we’re going to see other people. And not live together.”

Are you Really Doing Agile Development?

Fri, 14 May 2010 16:15:00 +0000

So recently at work I was asked help to make the company more “Agile.” Well I’m a developer first and process wonk second so I responded with my usual “How many of the 12 practices are you really following?” Which was met with a lot blank stares. Turns out the classic XP practices are not so easy to find on the internet anymore. It also turns out that the word “Agile” has been so successful that lots of people don’t know that XP stands for Extreme Programming. Now I’ll be the first to admit that “Extreme Programming” is a colossally stupid name, but what I like about XP and the original 12 practices is that they were controversial and easy to evaluate: Either you were doing them or you weren’t. What I don’t like about “Agile” is that it’s so broad and defined in such a “hand wavy” way that pretty much everyone can fool themselves into thinking that they already are fully Agile.

So to fix this and help my company start measuring their “Agility” better, I’ve decided to list out classic (They’ve changed over time – but not for me. And get off my lawn!) 12 XP processes. And my highly opinionated view of each of them. After that I’ll tell you how to score your “Agility.”

* Planning Game
Get requirements from customer, form them into (short) stories, estimate the time it will take, do them, measure velocity, examine failed estimates for clues as to how to estimate better, and repeat. A story should be a placeholder for a conversation – This only works if the customer (or surrogate) is highly available to the developers and the developers take advantage of that availability by checking in frequently during the development process. Note: Stories that take longer than a day significantly undermine many of the other parts of XP.

* Small Releases
Iterations should be 1-2 weeks (depending on how painful/expensive it is to organize an iteration). Releases should happen after every iteration if the application is easy to deploy (web apps).

* Metaphor
Nobody does this. Extra points if you do.

* Simple Design
This means different things at different levels. When kicking off a green field project a week of requirements gathering and design is good. For an individual story, a few minutes to an hour is good. Then you start writing code. Important: Design as you go. Stopping to spend a few hours with a whiteboard is allowed and encouraged. Do the simplest thing that could possible work until it becomes apparent that it will not. Then redesign. The idea is that at the beginning of the project you know the least about it – spread the design across the life of the project so you can make design decisions with more knowledge.

* Testing
Test first. Coverage above 80% for Java, 90% for Ruby (less exception checking in Ruby means easier to cover). Non-brittle, atomic, fast tests (unit tests should run under 5 min. Actually under 1 minute is best, but most think that’s impossible (spoiler: Its possible)).

* Refactoring
Short: Red, Green, Refactor! Long: Write a failing test, make it pass, refactor the code. This also means when you wander into a messy place you leave it a little better than you found it.

* Pair Programming
Above 80% of the time. Seriously. Tasks that seem like you can’t pair on them often mean that you aren’t using the right pairing techniques. Pairing is a skill: Get good at it.

* Collective Code Ownership
Anyone can change any code. Yes, they should consult whoever is knowledgeable about said code, but there are no parts of the code that only developer X is allowed to touch.

* Continuous Integration
Every check-in to the remote repository triggers a comprehensive suite of unit tests. The developers have to care if the build fails. For example: When the build fails, no one checks in until it is fixed. Also there should be some social ridicule for the build breaker (have to wear a stupid hat for the rest of the day, an obnoxious build breaker trophy placed on their desk, they have to buy donuts for the team, etc.). This all assumes that the build is reliable and that a failure means there is a real problem.

* 40-hour Week
Now called “Sustainable Pace” because “40-hour Week” tends to scare managers and end of project pushes are sometimes necessary even in XP. This should be no more than 2 weeks, have a clearly defined point at which the long hours will stop, and happen no more than twice a year. Actually, if it happens twice a year it means your estimation skills need to get better.

* On-site Customer
Doesn’t happen often for non-internal projects. Usually customer surrogates are used. These surrogates need to take pains to talk to real users often (not just the managers of the users) and be highly available to developers to answer questions.

* Coding Standards
Doesn’t matter what they are but the developers need to come to a consensus and stick to it. Consensus does not mean everyone nods yes in the meeting but ignores the standards when they feel like it.

So how do I score my Agility? Take the number of practices that you actually do, divide by 12, and then multiple by 100 – that’s your percent Agile. Do not give yourself a point if you don’t fully implement a practice. Have Continuous Integration but some of the tests fail randomly? No point. You pair “when necessary” which ends up being 40% of the time. No point! Run iterations but don’t estimate your stories? No Point!!

This is take a hard look at yourself in the mirror time, not some hippie love fest.

Lone Star Ruby Conf 2009 Day Two Afternoon Sessions

Sun, 30 Aug 2009 23:51:00 +0000

More Lighting Talks!

A lot of people skip the lighting talks, but I find that some of the best stuff at any given conference comes out of the these intense 7 minutes sessions.

Jared Ning: “Ruby Without Borders”
Matthew Todd gave a talk last year asking for people to came to Tanzania and help him code in Ruby. Jared did and he loved the experience.
He mentioned a very useful TextMate short-cut:
If you have a bunch of files and folders open in the project drawer and you click the chevron (the triangle thingy) to close a top level folder then when you open it back up TextMate remembers the state of what folders were open and closed underneath. But sometimes you want to close down all the folders underneath. Option clicking on the chevron will do that. Also, option clicking on a closed chevron will open up all sub folders. I’ve been looking for this.

Check out Ruby Without Borders.

Jeffrey Taylor: “Fast Multi-protocol XML Parsing”
Jeffrey’s project had to read a lot of RSS feeds and it was super slow even with the fast ruby xml parsers. So he rolled his own, which can be found here:

http://abluz.dyndns.org/fast_xml_parsing.rb

Jeffrey used the sax model to do super fast xml parsing (10 times faster than nokogiri - he claims). He does admit that the code has a Flog score of over 500 for one method.

Hal Fulton: “Reia: The Next Big Thing”
Reia is an attempt to combine Erlang and Ruby. If you want the cool concurrency of Erlang with Rubyish syntax, take a look.

Yehuda Katz: http://github.com/wycats/bundler
A true dependency resolver
From the README:

Bundler is a tool that manages gem dependencies for your ruby application. It takes a gem manifest file and is able to fetch, download, and install the gems and all child dependencies specified in this manifest. It can manage any update to the gem manifest file and update the bundled gems accordingly. It also lets you run any ruby code in context of the bundled gem environment.

So basically it’s a better way to vendor your gems with your project. Check it out if you’ve had trouble using other systems.

Before Matz’s Q and A, Jim Freeze got up and talked about Lone Star Ruby Conf’s attendance over it’s three year span:
2007 - 200
2008 - 282
2009 - 230ish

Which is right around where they want it. Small enough to be intimate, large enough to be interesting. And that size group fits nicely in the Norris Conference Center.

Btw, the network was outstanding this year. They bought something like 6 wireless routers and although sometimes it was sluggish, it remains some of the best wireless I’ve had at a conference. Usually the high tech load crashes the network within minutes.

Matz Q and A
The two answers that stood out to me were when he said that the Perl $ variables were the thing he regrets the most about designing Ruby. Also, when I asked him about how often he gets to write in Ruby he said that the largest Ruby program he ever wrote is 2-3000 lines of ruby code. It was a mail client with Emacs front end. Generally he uses it for scripts and such. I find it amazing and a little heartbreaking that Matz doesn’t get to program in the language he created and clearly loves.

Encoding Domains - Rich Kilmer

First, there was a lot of buzz about Prezi at the conference. Prezi is a canvas presentation tool that so impressed Rich Kilmer that he learned it the night before and wrote his keynote in it. In Prezi you put all your text, videos, images, sound files, and whatever in one huge canvas and then tell give it a bunch of waypoints and zoom levels to follow. It’s pretty bad-ass.

This was Rich’s first keynote ever. Which is kinda hard the believe as he’s been doing amazing things with Ruby since the dawn of time (which is 2000, btw). In it he discussed how software libraries don’t have value, they have potential. What has real value is encoding domains. He told an interesting story about how he encoded the domain of a massive military project mostly by sitting down with some experts in the field and figuring out how they would like the DSL to read. Once he had captured all the information they could enter in to their logistics system in syntactically correct Ruby, he went about making it work. He called it syntax driven development and it worked so well that 5 years later he still gets calls about the prototype he wrote in two months. Apparently they are still trying to create the real product in some other language.

Well, the perfect storm of conferences (Agile 2009, Software Craftsmanship North America, and Lone Star Ruby Conf) is now over and I’m looking forward to actually writing some Ruby tomorrow. I need the rest.

Lone Star Ruby Conf 2009 Day Two Morning Sessions

Sat, 29 Aug 2009 17:52:00 +0000

Last night Matz gave a keynote entitled “Why do we Love Ruby?” He talked about how Ruby embodies Quality Without A Name (Qwan). Here’s a description of Qwan.

I think it might be time to retire this talk, as I’ve seen it more than a few times before. I’d much rather hear about interesting problems he’s solved while designing Ruby or meditations on where the language is heading. A humble suggestion from a guy who’s very grateful for Ruby.

TDD: More than just “testing” - Evan Light

Evan started out his talk by pointing out that lately we’ve been focusing more on tools and techniques more than the goal of testing. Testing is not the end, it is a way to help you get to the goal of a well designed application.

He did a live BDD coding demo - which is very courageous. I’ve seen way too many presenter’s brains melt under the hot lights and audience pressure. He managed to pull it off and show some Behavior Driven Development. Evan likes to write out his series of specs/tests before he writes any code. It’s sort of a high level sketching out of the design. Then he works on getting them to pass. Watching him work I was struck by how much good BDD is a conversation between the tests and the code - going back and forth. Sometimes the tests push bask and tell you the code needs to change, but just as often the code can push back on a naive test.

At this point I got up and gave my talk. The crowd was way more into it than when I gave the same talk a few days ago at Agile2009. Later people kept coming up to me and thanking me for the talk and metric_fu. Felt pretty good.

Lightning talks:

How to Get More Women (in Programming) - Sara Brumfield

Anti-patterns:

Impostor syndrome - a lot of women feel like they are impostors so they don’t go to conferences. They don’t realize that just about everyone feels that way.
Voice of unquestionable authority - The one guy who asks the really hard intense question. This action can intimidate women a lot more than men.

Suggestion: Offer women’s t-shirts at you conference. Little things like that encourage women to attend.

The “Pipeline Problem.” Math classes filter out some women, programming classes filter out more, the job itself keeps filtering. Sara feels that we’ve been trying to solve the pipeline problem for years, but she feels it’s not working.

Sara posted a more detailed description of her thoughts on the subject.

Prawn - Gregory Brown

Prawn is a Ruby gem for creating PDFs.

High level interface for basic reports.
It’s the fastest pure ruby pdf generator
Prawn is pure Ruby with no tricky dependencies so it runs on all the Rubys
M17I is a top priority

The PDF specification is 1300 pages so it’s been interesting. 1.0 of Prawn should come out soon.

HTML5 & CSS3 - Dallas Pool

The combination of HTML5 and CSS3 manages to move a lot of the dynamic view stuff out of flash, javascript, and photoshop and into… the view! Looks pretty cool.

If you want to know more checkout a presentation he gave here: http://www.scribd.com/doc/19111526/HTML5-Presentation

CodingMentors.com - Mark McSpadden

Mark wants you to get involved with mentoring developers in specific areas or to get mentorship. You can commit to remote mentoring and limited hours so don’t be scared.

Spork - Tim Harper

Spork is a gem that forks the linux process so you can run rails specs faster. Basically you don’t have to load up the rails environment every time you run a test (which is why tests that claim to take 0.023 seconds really take 5)

Time for lunch!

Lone Star Ruby Conf 2009 Day One

Fri, 28 Aug 2009 22:26:00 +0000

I have to admit that this morning I was not looking forward to seeing another day of presentations after 3 days of Agile 2009 and 1 of Software Craftsmanship North America, but almost immediately upon entering the Norris Conference Center I felt welcomed. Part of this is that Jim Freeze, the LSRC head organizer, is a heck of a nice guy who recognized me in the hallway and said hello but also that when I sat down I noticed that every table had plenty of power for computers. Awesome. At Agile there was no power in any of the rooms and very limited access anywhere else. I type up my notes on the presentations as they happen (and I notice a lot of others do the same) so I spent much of my time at Agile glancing at my diminishing power and hoping I could make it through the next session. I know a lot of people hate open laptops at conferences but studies have shown that if you give some people something to do with their hands while they listen to a presentation they consume it more effectively. I know it can seem annoying to have the person next to you furiously IM-ing all his friends but that may just be his or her way of listening.

Something interesting - Dave Thomas

Ah to be so famous that don’t have to bother to title your talks. Despite what seemed like a lack of preparation, once he got up there his talk seemed quite polished and interesting.

Dave mentioned early on that it’s his tenth year using Ruby which is amazing for a man who has “the attention span of a gnat.”

Why?

First of all, Ruby is a Multi-paradigm language. Ruby can do procedural, prototypes, objects, and even a passible interpretations of functional all in one somewhat unified whole. They idea is to let you, the programmer, do what you like.

“Ruby thinks the way I think” - which means it’s imperfect. “It’s gloriously imperfect.”

Dave then showed a bunch of quotes from famous people talking about how the sterility of perfection can hinder creativeness. He ultimately concluded that if their is no ambiguity, there is no room for you.

An example of Ruby’s ambiguity:
def foo x
puts x
end

foo { :e => 2 } # Syntax error!
foo :e => 2 # Totally fine

Ruby’s interpreter has some quirks - there are heuristics that make your code read naturally and work 99% of the time. So, Dave concluded, some ambiguity is worth it.

Ruby keywords are loose. For example, is ‘begin’ is a keyword? Check this out:

> c = Object.new
=> #>> def c.begin
> puts “hello”
> end
=> nil
> c.begin
hello
=> nil

Wacky.

Ruby’s view is there is no right way.

The progress in Ruby is Messy. The size of the Ruby tarball is going going up exponentially while the number of methods is going up linearly. And Ruby 1.9 is very different from 1.8x. The benefit to breaking some backwards compatibility is new and powerful features keep entering the language.

The People are Messy.

Ruby was the wild west in the beginning with a small true community. But then someone found Gold in them thar hills. Rails came in and all of a sudden the group grew by leaps and bounds.

Dave was physically sick reading Zed’s rant (in which Zed Shaw said some none too nice things about Dave). It was 9 months before he slept properly again. Dave thought he was part of the Ruby community. But Ruby is not a singular community anymore. To say Ruby is a community now would be like claiming there was a screwdriver community. Ruby is a resource around which communities gather.

In conclusion: Messy can be fun.

Programming Intuition - Glenn Vanderburg

Last year Glenn gave a talk called ‘Tactical Design’ in which he advocated focusing on three things for the beginning programmer:

Do One Thing
DRY
SLAP (Single Level of Abstraction Principle)

Last year’s talk was all about what you can do. This talk was about how you think.

What makes a programmer good is not a command of the syntax, it’s more like art. Great programmers all talk about code as if they can touch and smell it. Large portions of human brains are designed to use 5 senses.

Glenn showed Bobby McFerrin at the world science festival teaching the audience the Pentatonic scale. Then he showed a visualization of Frederic Chopin nocturne opus 27 #2. The point being that music/computation/thinking are all would up together.

At one of Paul Graham’s shops they had some mechanical gauges to tracking networks traffic. He could hear the gauges moving and knew when there is a problem without ever having to get up and look at the display.

Glenn was saying that we need to cultivate a sense of code. The best programers seem to use all the parts of their brain including the 5 senses. Quick feedback is essential to this.

Module Magic - James Edward Gray II

So when confreaks post this talk online go check it out. There was way too much code on the screen for me to even attempt to copy it all down. James showed off some pretty cool things you can do with modules. Check out his slides or the full talks and meditate on the code. What I really like about James’ presentations is that he knows how to start off slow. He has a real teacher’s gift for bringing an audience along.

Towards the end of the talk James said that he feels that modules can replace a lot of what we use inheritance to do now and without inheritance’s side effects.

Update, he’s posted the slides.

Succeeding with Rails - Chad Pytel

Chad is one of the founders of Thoughtbot and he went through some Thoughtbot’s values and strategies that have guided them.

Hiring process:
1. Initial Resume and code review
2. Initial phone call - high level (who you are, what your goals are)
3. Second techincal interview.
4. Week long immersive trial. Payed.

Core hiring principles:

Look for the best Skill and Attitude at a Salary you can afford.
They hire all technicians (coders)
No subcontractors or outsourcing

Clients:

Do the best you can for the client
Be proactive (over communicate). Have the hard conversations early.
Set expectations and keep setting them. See above
Be nice. And fire the not nice customers.
Work the way they want or no deal.

Client interaction:

Their project planning form is online.
They use highrise for managing client information.
Use master services agreement (how we work and how much you pay us) and set out the first several iterations
Appoint a project lead (PM/CP)

Development values:

Practice sustainable development - 40 hour work week
Design first (build the user interface)
TDD
Iterative development - use Pivotal Tracker
Focus on time instead of money
People should do what they enjoy
Always at least 2 developers
Rotate people in and out of projects

Everything you do is marketing

Open Source:

A part of their culture
Payback for all the open source they use
It Legitimizes what they do
Gives them crowd-sourcing support
They keep the quality of their open source high by using it internally before it goes out

Why did they get into products?:

Consulting scales with people - but they don’t want lots of people.
Products scale without large groups of people

They focus on solving problems that they have and products they enjoy building

Checking under the Hood: A Guide to Rails Engines - Mike Perham

This was interesting talk where Mike went over Rails Engines, which are basically like a plugin that can have views and controllers.

Note: The real application code always wins over the engine in cases of name or file collision. You can use this to your advantage with a view you need to change.

There are no way to add migrations

To avoid model name collisions use name-spacing (Foo::User instead of User).

‘helpers :all’ will not load engine helpers

Static assets need to be copied over from the engine to the app

Dataflow: Declarative concurrency in Ruby - Larry Diehl

Dataflow is a ruby gem that handles some interesting concurrency problems in Ruby.

In your model:

class Foo
declare :my_var
def initialize
unify my_var, 1
unify my_var, 3 # kaboom
end
end

the second time you try to change my_var, it blows up.

local do | x, y |
Thread.new { unify x, “blah”}
Thread.new { unify y, “argh #{x}”}
y.should == “argh blah”
end

No mater how long the threads take a call to y will work in the above code.

There is a need_later method that will go off and do it’s thing. If you try to call a method on the object returned by need_later it will either wait (if its not done) or return it (if the thread has finished).

FutureQueue, may come in at some point. With FutureQueue you can pop from it before you push to it.

Larry made inspect not block for debugging purposes.

Agile Conference 2009 - Day Four

Fri, 28 Aug 2009 03:43:00 +0000

Agile’s Too Slow: Developing a Facebook App for the Obama Campaign - Andy Slocum

Andy worked on the Facebook application for the Obama campaign. In two months they build a Facebook app that could assist users in encouraging their friends to register and vote. There where only two people on the project and they came in with the common ideas of XP: continuous integration, weekly iterations, story estimates, developer testing, etc.

Except that all of the above was way too heavy for the Obama campaign’s pace. They’d have an Iteration planning meeting on Monday and by the Tuesday it was out of date because so much had changed. Also Facebook apps don’t really run very well outside of Facebook so they could only write tests around code that was completely isolated from the Facebook stuff. So they basically moved to a lean process with priorities set at the standup meeting every day. They also ditched the idea of continuous integration as there were only two people to integrate.

An interesting part of the presentation was about the lack of privacy in Facebook. Any Facebook app can get the birth date, state of residence, and name of any user and all their friends. So the Obama app could use existing voter registration databases to figure out if a user’s friends were already registered.

They deployed the application multiple times a day. Their load testing was asking one of the stakeholders, who had over a thousand Facebook friends, to test out the new features.

The Ogre and the Wimp: Cleaver Influencing Tricks - Help the Most Reluctant Teams – Anda Abramovici

This talk was about how to introduce agile practices to particularly stubborn developers. Anda was a consultant on project where she had a bunch of developers who didn’t want to pair or use TDD. She conspired with an upper level manager known for her toughness to come in and play the Ogre while she played the wimp. She would join the team and pretend to be just as unhappy and resistant to TDD and Pairing as the rest, but then the Ogre would come in and demand that the team increase it’s test coverage. Then she’d feign resistance while helping the developers respond to the ‘crazy’ demands of the Ogre.

Later they conspired to move a bunch of equipment into Anda’s workstation so she had no place to sit. So she shared an station with another developer. Eventually they started pairing.

They also recommended using free drinks to help the wimp bond with her new team.

All this seemed a little dishonest to me and has a large chance of backfiring if the wimp was ever found out as an agile advocate. It did, however, work for them.

Slow and Brittle: Replacing End to End testing – Arlo Belshee and James Shore

This was a workshop where we discussed the problems of End to End testing (they tend to be, uh, slow and brittle) and possible solutions. I had to leave to catch my flight to Austin (I’m on a plane to Lone Star Ruby Conf as I type this) before the end so I’m interested to hear what they came up with. One of the answers was the idea of giving up on trying to test all paths in an End to End test and focusing on unit tests and an abbreviated ‘smoke test’ suite that is forbidden to go above 5 minutes. There was quite a lot of passion in the room.

OK, I’m exhausted. Time to hit ‘Publish Post’ and relax for a bit as tomorrow will be my third conference in as many days.

Software Craftsmanship North America Conference

Thu, 27 Aug 2009 04:25:00 +0000

This was the first ever Software Craftsmanship conference and it was put on by my company Obtiva and 8th Light (a company I have pretty strong ties to). It was a pretty big day for the Craftsmanship movement and our little companies.

Software Craftsmanship Manifesto is:

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

Not only working software, but also well-crafted software

Not only responding to change, but also steadily adding value

Not only individuals and interactions, but also a community of professionals

Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

The idea is to build on the Agile Manifesto toward better software and the community that surrounds it.

Swimming Upstream, Sprouting Legs, and Running Free – Ken Auer

This is by far the most provoking talk I’ve heard in awhile. At the beginning Ken announced that he wasn’t going to be politically correct. And as the talk went on I kept wondering what that was going to be. Would he go after women drivers, Mexicans, president Obama…? I kept waiting for the other shoe to drop.

Ken had a fairly happy childhood where he desperately wanted to be a baseball player. But, alas, that dream doesn’t often come true. But he was good at math so he played a lot of “Strat-O-Matic Baseball” which is a game where you use the stats of baseball players. He developed a series of algorithms so that he could beat all his friends. This, of course, lead to a career in computers.

The other theme in Ken’s life is religion. This, apparently, is ‘not politically correct’ part of the talk. I gotta say that it’s takes a lot of chutzpah to talk about religion at a software development conference. I have never heard somebody say “Do you accept Jesus Christ as your personal saviour?” at any conference before, but I have now.

Ken attended church all throughout childhood going there “whenever the doors were open.” But he feels that he was sort of passively religious during that time. He didn’t know much about the bible or question any of the church’s practices.

In Ken’s software life he had his eyes opened by the Object Oriented revolution when he attended the first ever OOPSLA in 1986. He and his company got into OO and Smalltalk very early on and produced a bunch of working software before anyone could tell them that Smalltalk wasn’t for ‘real’ software. It’s funny how often people succeed because they didn’t know it was supposed to be hard.

Also, around this time, Ken started attending Bible study groups and thinking more about his work/life balance. He spent a few years ruminating about how to affect a change and eventually came up with the idea that he would start his own company “Role Model Software.” Over the years Ken has:

Mentored a bunch of apprentices
Built a huge house where he lives, works, and teaches his children (he’s a homeschooler)
Become an elder at his church
Created a lot of working software

Ken seems pretty happy with the current of his life. He’s managed to combine a bunch of the formerly separate aspects of his life into one unified whole.

Late in Ken’s presentation he ran out of time and had to skip past a bunch of slides (I think I saw Darwin in there somewhere) and wound up with a slide that said “I don’t believe we came out of the mire. But we need to.” Which I take to mean that he doesn’t believe in Evolution but that he feels that our current software situation is a bit of a mess and needs to be changed.

Again, easily the most provoking conference presentation I’ve ever seen.

Self Eduction and the Craftsman - Michael Feathers

Michael’s talk was mostly a list of things he thought that people who were self taught or apprenticed might not have learned (because they weren’t in a formal Computer Science program).

Things you need to know about:

Big-O, Little-O, and Theta Notation
Covariance & Contravariance
Types
Objects are Closures
State Machines
Regular Expressions
Turning Machines
The Halting Problem - limits on verifiability
Worse is Better (just good enough) (less features is easier to debug)
Redundancy is not strength - “It’s a weird industry where we are the things that introduce problems into a working system” “The same spec given to different teams tends to produce the same bugs”
Security on Sand (“On Trusting Trust”) Security can only be so good
Location Transparency is a Myth
Objects are Clay

Books that you might not have read:
Structure and Interpretation of Computer Programs
“Designing Object Systems” Cook and Daniels (Syntropy)
“Graphs: Theory and Algorithms” Swamy (Graph Theory)
“Compilers” Aho, Sethi, Ullman (The Dragon Book)
Discrete Mathematics - Anderson
Theory of computation - Michael Sipser

Observations from an Old Warhorse - Fred George

This talk, by the highly engaging Fred George, was a series of completely unrelated observations from a guy who’s done a lot of Agile and non-Agile projects.

What is considered a big application?
1970 - 10K Lines
1980 - 100K
1990 - 1M
2000 - 10M
2010 - 100M

But effort seems to rise and fall to build big applications because we keep getting advances in writing software and all advances have this in common: Simplify the problem. Get rid of duplication.

Discard to Make Languages more Robust:
goto
Tektronics School of Objects: ‘if’ is suspicious ‘else’ is almost always wrong.

I’ve heard this ‘If/Else’ claim before but without any examples, it’s hard to understand how to purge my code of all the ‘Else’s.

(Rails - TDD) == Productive VB
If application is less than a page of code would you..
- write tests?
- pair?

Fred doesn’t write test or pair on micro projects.

Feedback => Quality
1. Frequent releases
2. iterations
3. Stand ups
4. Tasking Cycle 1-2 hours (all the way through a story in 1-2 hours? This guy is big into Lean.)
5. Pair Programming

Lean: Identify and Eliminate Waste
What are you making?
How do you make it faster

Story life cycle.
Finger charts are useful.

Iterations are dead
From scrum’s couple of months
to xp’s 2-3 weeks
to ThoughtWorks 1 week
to Daily (part of the stand up)

Fred uses a Kanban board to track stories in progress.

Developers pick up the most important thing next. If management wants to have a meeting – let them.

Continuous Deployment

Holy Grail of Software Development: Short lead times

Right now on Fred’s team: Over 50% of stories are deployed within 2 days of the story surfacing. Which reduces the need for branching.

Grand Unified Theory of Software Design – Jim Weirich

This was an interesting attempt to find a unifying principle underlying all software principles (Don’t Repeat Yourself, Single Responsibility, etc.). Sort of like how physicists are looking for a single underlying law to the four fundamental forces.

Jim thinks that using ‘connascence’ he can derive many of the principles. Connascence is defined by thefreedictionary.com as:
Con`nas´cence
n.
1. The common birth of two or more at the same tome; production of two or more together.
2. That which is born or produced with another.
3. The act of growing together.

For software connascence is when changing one piece of software requires a corresponding change in another place.

As the distance between the software increases we want to use weaker forms of Connascence

A weak form of connascence is connascence of name
def foo(bar)
bar + 1
end
if I change the name of ‘bar,’ I have to change the next line too. If I change the name of the function ‘foo,’ I’ll have to change all the places that call the ‘foo.’ So, even within connascence of name there are levels.

Connascence of position is when order matters (worse than connascence of name):
def foo(a,b,c,d,e)
….
end

People calling this method have to remember the order of the arguments. A common solution in Ruby is to use a hash to move to a lower form of connascence (connascence of name):

def foo(options)
…
end

so you can call the method like so:
foo(:position => 54, :height => 10, :width => 20, :depth => 3, :solid => true)

A common connascence of meaning problem is magic numbers. If many places have to know that -999 means a bad response this is connascence of meaning.

Connascence of timing is race conditions

Connascence of Execution is when the order of the steps matters.

Unfortunately I wasn’t able to get down all his examples of how Connascence is the common link for all software problems (and I think this is a work in progress) so hopefully Jim will write this up in more detail some day.

What if Bacteria Designed Computers? – Ward Cunningham

Ward Cunningham is legend in Agile, XP, and computing in general so he can pretty much talk about what he wants and everyone will applaud at the end. Today he was talking about buying tiny IC chips, wiring them together, and getting them to communicate like cells. By doing this he was able to build completely new and seemingly complicated devices (such as a flag waving robot) simply by swapping some parts around. Was this a powerful metaphor for having a reusable tool set or just a guy talking about geeky stuff? He never came out and made it explicit. I enjoyed the talk none the less.

The Business of Craftsmanship - Kevin Taylor, Micah Martin, and Carl Erickson

The owners of three craftsmanship oriented shops got together and took questions from the audience. Carl Erickson founded Atomic Object in 2001 in Grand Rapids, MI. Kevin Taylor started Obtiva in 2005 and Micah Martin created 8th Light in 2006.

This discussion rather quickly became about how the Apprenticeship programs work at each company. At Atomic object it seems to be a summer thing for college students and it continues from summer to summer as long as it’s working out. If it goes well, it leads to a hire. Obtiva brings in apprentices and the apprenticeship may continue for a year or two before they are moved out of the apprenticeship. Apprentices can be moved into billable work whenever they are ready which could be a matter of weeks or months but no more than 3 months (it’s a paid position so the company can only afford to carry someone for so long). At 8th Light apprentices are assigned to one of the higher level consultants and are non-billable for 3 months. After a series of challenges they are either given another 3 months, promoted, or let go.

All three companies have experimented with different types of overseeing the apprentices and have generally found that someone needs to ‘own’ the mentor-ship of an apprentice otherwise, as Carl Erickson said, “You’d see two apprentices off in a corner working together” and that didn’t work.

At that point I had to run off and give my “What’s the Right Level of Testing?” talk at Agile 2009. Since the two conferences were held a city block apart, the commute wasn’t too hard. The talk went pretty well, there were about 40 attendees but they mostly seemed kinda sleepy. 4 or 5 people were pretty engaged though and seemed to enjoy it. 90 minute presentations are the rule here at Agile so when you have a 45 minute talk two of them get squished together into a 90 minute slot with no time for transition. So to attend my talk someone would have to see the 45 minute talk before me and hang around, leave another talk mid-stream, or sprint from one 45 minute talk (which could be in another tower) to mine in zero seconds. So with these rationalizations firmly in place, I feel good about the turnout and how it was received.

Tomorrow I’ll be able to see some of talks but I’ll have to leave early to fly to Austin to present at Lone Star Ruby Conf.

Agile Conference 2009 - Day Two

Wed, 26 Aug 2009 01:14:00 +0000

“I Come to Bury Agile, Not to Praise It” – Keynote by Dr. Alistair Cockburn

So this is one of those classic keynotes where the title has little to do with the content of the talk. I did enjoy the keynote, very much so, but only the first 10 minutes were devoted to Agile’s ‘Death.’

Alistair came out following a bagpiper playing “Amazing Grace” – like a funeral. I gotta admit that it was a pretty nice entrance. Then he gave a modified version of “I came to bury Caesar, not to praise him” speech from Shakespeare’s Julius Caesar (full text here: http://alistair.cockburn.us/I+come+to+bury+Agile%2c+not+to+praise+it). His point was that Agile started out as something for small, co-located, groups and was pretty rare. It is now very prominent and used in quite large organizations. So it has gone through a bunch of changes despite the fact that we call it by the same name. Not quite a ‘death’ but “Isn’t it weird how we still call Agile the same thing even though it’s changed a lot since 2000?” isn’t a catchy title for a keynote.

Then he spent the next 70 minutes talking about the 4 pillars of Agile:
1. The finite co-operative game
Goals of the game:

Deliver the software
Setup the next game

Moves you can make:

Invent
Decide
Communicate

The situations almost never repeat

2. Craft
Pay attention to skills and to the medium

People learn in 3 stages:

Shu: learn a technique
Ha: Collect techniques
Ri: Invent/blend techniques

Beginners tend to get stuck in the ‘Shu’ box. Alistair claims he spends most of his time trying to kick people out of the ‘Shu box.’

3. Lean processes
Aim for continuous flow
Watch your queues (where the bottlenecks are)

4. Knowledge Acquisition
If you integrate all the time (or release) you can have knowledge lead cost (spending on the project) then you can trim the tail of the project to deliver on time or extend to get a little bit better value. If you integrate late the options are: Deliver late or work overtime.

At one point he claimed that there are studies that show how expensive distance is to a company. 12 people not pairing costs 100,000 per year and goes up to $300,000 if they are on different floors. Taking a look at his slides, I think this fact was taken from the book “Managing the Flow of Technology,” but I’m not sure.

Link to the slides.

Idea Factory - Brian Marick, David Carlton

This was an attempt to relate 3 ideas of how scientific ideas are adopted to Agile.

The “Trading Zones” idea is from a book by Peter Galison titled “Image and Logic: A Material Culture of Microphysics.” The idea is to setup local areas where separate groups working on the same project, but do drastically different things, can get together and possibly form a common language.

Packages are a way of, well, packaging your ideas so that they are easy to assimilate and adopt.
Example: JUnit made it easy to do TDD. (That’s kind of what I’m trying to do with metric_fu and code metrics).

Actor/Network theory is a study of “how does a fact become a fact?” A successful idea will, over time, go from:
Something that is cited conditional in publications. Ex. “Watson and Crick propose that DNA has a double helix structure”
to
Being cited without question. Ex. “Watson and Crick proved that DNA has a double helix structure”
to
Being assumed. “Given the double helix nature of DNA…”

XP: My Greatest Misses 2000-2009 - J. B. Rainsberger

This was another interesting talk by Rainsberger. He discussed his greatest failures as an XP/Agile coach/trainer. Almost all of his problems were caused by his failure to consider the people part of software. His ideas, processes, and plans may have been spot on in his own mind but they were often interpreted by others as challenges or insults or worse.

The breakthrough for him was when he finally started reading the works of Gerald M. Weinberg and Virginia Satir and learned about the human side of consulting. When J.B. speaks he thinks about how what he said will be interpreted. And when he hears something jarring or offensive he tries to figure out what an alternate and generous interpretation might be for the comment.

He wrote up an article about miscommunication entitled: Don’t Let Miscommunication Spiral Out Of Control

Improving Obama Campaign Software by Learning from Users – Paul Baker, Billy Belchev, and Zack Exley

Every 4 years the election software budget goes from 0 to 800 million. Which can be hard on the software. Paul Baker and Billy Belchev were hired to evaluate a bunch of software used by Obama volunteers during the campaign to track canvassing and phone banking efforts.

They did this in 3 ways:
1. Contextual inquires (interviews)
1-5 users per task
30-45 min per interview

2. Lightweight usability testing
1-3 user per task
15-25 min per session

3. Collecting artifacts
Taking pictures and video of the environment, people, and software interacting (or, in some cases, failing to interact)

Basically they went out into the field and talked to people as they used the software. Amazing how this simple strategy quickly (they spent 9 days in the field) finds huge problems with the software.

Example quote: “The way I would really do it? I’d write her name down and call her later.” Instead of use the cumbersome tracking software.

Lots of offices used Google Docs to store information because an overly restrictive permission system wouldn’t let key members see data and to store specific fields the software couldn’t hold.

All the sticky notes on the wall, computer, and stuff on the whiteboards will tell you what the software isn’t doing (or doing well)

They used a lot of video clips with people talking about how they worked around the bad parts of the software.

Pretty good day, all in all. Tomorrow I’ll be attending the Software Craftsmanship Conference sponsored by my own company Obitva. Then I have rush back to Agile to give my talk “What’s the Right Level of Testing?” at 4:45. Seeing the caliber of talks here I’m pretty proud and excited to be a presenter.

Agile Conference 2009 - Day One

Tue, 25 Aug 2009 02:29:00 +0000

It’s been 5 years since I attended the granddaddy of Agile conferences and I have to say that the first day was damn good. My notes my be a little rough, so I apologize in advance, but if I don’t type and post them on the day I pretty much never will.

Workflow is Orthogonal to Schedule – Mary Poppendieck

Mary started out with an example of on time development from 1929: The Empire State Building.

In September of 1929 they started cleaning the constructions site of previous buildings and by May 1st of 1931 the building was finished and open to the public. It was exactly on time and 18% under budget.

Cash flow was king: Every day of construction was another day before they could start getting their money back.

How did they do it? They focused on flow. They thought of the project as “A marching band going through the building and out the top.”

Key success factors:

The owner, architect, and builder got along well and worked as a team.
They had a deeply experienced building team.
There was a lot of focus placed on the key constraint: Material Flow. 500 trucks a day but only arrived with materials for the building, but they only had a few days of storage on site so everything that arrived had to go up and onto the building.
Decoupled the work: The systems (windows, floors interiors) were designed to be independent
Cash flow thinking: Every day of delay coat $10,00 ($120,000 in today’s dollars)
The schedule was not derived from the design. The design was put in place to meet the constraints of the schedule.

Mary used this as a shining example of Lean development in the pre-computer era. So how do you run a software project with the highest degree of efficiency possible?

Establish high level system goals (what do we really need).
Involve those who understand the details early in the process.
Use teamwork based on trust. Contract thinking increases costs 30-50%.
Reduce complexity wise wise decoupling
Understand and manage the constraint
Features can be big but dev stories must be small (1-3 days at most)

At this point she compared Kanban to Iterations. This article describes Kanbab better than I could do it justice: http://www.agileproductdesign.com/blog/2009/kanban_over_simplified.html

Schedules (are way different from flow)
1. If you have a level work flow you have both predictability and control without a schedule
2 Schedules based on experience are reliable
3. Schedules summed up from task breakdown are not.
- without experience, task break-down/sum up schedules are a hypothesis
- without slack, task breakdown/sum up schedules are deterministic and do not allow for normal variation
- attempts to remove normal variation from a system cause oscillation

Asking for feedback means you’ll get it. You can only commit to 50-70% of what you want because you will get feedback that takes over the rest of the time.

Small batch size means you can get 80-90% utilization – an iron law of math. Utilization above this range causes thrashing and wastes lots more time than it gives you.

Optimize Throughput, not utilization

She covered a lot of interesting stuff in her talk and I feel like I’ve not really captured the flow and force of her arguments. Mary is one of the best Agile speakers around and I recommend taking any opportunity to see her talk.

What Do Agile Coaches Do? – Liz Sedley and Rachel Davies

This was a tutorial were the audience broke into groups and discussed Agile coaching. The presenters posited that there are 3 types of coaching:

Consultative - Trying to change process
Motivational - Encouragement
Education - Helping understanding

We looked at some common problems that Agile coaches face and discussed solutions. I thought the conversations I had were pretty interesting but I would have liked to hear more from Liz Sedley and Rachel Davies about there experiences in Agile coaching as they had just written a book on the subject.

Integration Tests are a Scam – J.B. Rainsberger

This was a pretty awesome talk and I totally owe my attendance to Colin Harris, as he recommended a blog post to me about just this topic by just this author.

J.B. defines integrations tests as any tests that test more than one class.

Why is this bad? Well he spent 90 minutes making a very solid case, but let me try to sum up:

First he started off by saying that he’s talking about developer tests and not customer tests. In other words, he cares about tests you run before you check in not huge regression suites used by Q.A.

1. Integration tests are slow. Slow tests will eventually become ignored tests.

2. Testing a group of, say, 3 interacting objects creates a ridiculous amount of tests you need to write. If there are 5 paths through the main object under test, but the methods it uses in another object have 3 and 10 paths respectively then you should write 3 X 5 X 10 = 150 tests to exhaustively try out all the paths. However, if you test each object in isolation then you need to write 3 + 5 + 10 = 18 tests to get the same coverage of paths. You are far more likely to write 18 tests, then 150.

The way J.B. does this is to create interfaces for all his objects that have complex behavior (He likes Domain Driven Design so he tends to refer to these objects as ‘Services’) and then use ‘Doubles’ (mocks and stubs) to eliminate the need to use the real objects. Now when you use doubles, you need to make sure that the thing you’re mocking on one end really does return what you think it does on the other.

When I asked him about having setups with lots and lots of mocks he claimed that if you’re having trouble mocking things in your tests you probably have problems with your models. Sigh. He’s probably right. For instance, if you need to have a mock that returns a mock which returns a mock you probably have an object that is breaking your layered design.

Of course when I test in Rails I have to do this all the time. Part of the reason is that good design is way hard and I’m still learning. But the other is that Rails has ActiveRecord which is an entity that is responsible for it’s own persistence and that is something that J.B. and a lot of people I respect look down upon.

Rainsberger gave me a lot of things to think about which is pretty much the highest praise I can give a session.

Point Inside a Polygon in Ruby

Wed, 22 Jul 2009 23:07:00 +0000

Recently my team needed to find out if a point was inside a given polygon and, as usual, we found some code on the internet that did what we wanted. And, as usual, there wasn’t much explanation as to what it was doing. Here’s the code (after we converted it to Ruby):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def contains_point?(point)
c = false
i = -1
j = self.size - 1
while (i += 1) < self.size
if ((self[i].y <= point.y && point.y < self[j].y) ||
(self[j].y <= point.y && point.y < self[i].y))
if (point.x < (self[j].x - self[i].x) * (point.y - self[i].y) /
(self[j].y - self[i].y) + self[i].x)
c = !c
end
end
j = i
end
return c
end
end

So, you know, it works and I’m grateful to somebody for posting it but we had no idea how it works. So we wrapped 2 tests around it (one where the point is not in the polygon and another where it is). But that one method had a Flog score of 80 – it was killing me. Also 2 tests where nowhere near exercising all the possible paths through this code and since it was completely un-obvious as to what was going on this was just a bug waiting to happen.

I decided to refactor this method so that it made a wee bit more sense. I did a little bit of reading on the internet about the ray casting algorithm, got out a pen and paper, and figured out what the hell was going on. Before I go on to explain how I changed the code, I should probably explain that we are using GeoRuby to give us a bunch of geometric classes and we added the contains_point? method to GeoRuby::SimpleFeatures::LinearRing. Inside a LinearRing instance self[0] is the first point in the polygon, self[1] is the second, and so on. Also, point.x gives you the x value of the point. Oh, and self.size returns the number of points in the polygon

Here’s the refactored code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

def contains_point?(point)
contains_point = false
i = -1
j = self.size - 1
while (i += 1) < self.size
a_point_on_polygon = self[i]
trailing_point_on_polygon = self[j]
if point_is_between_the_ys_of_the_line_segment?(point, a_point_on_polygon, trailing_point_on_polygon)
if ray_crosses_through_line_segment?(point, a_point_on_polygon, trailing_point_on_polygon)
contains_point = !contains_point
end
end
j = i
end
return contains_point
end

private

def point_is_between_the_ys_of_the_line_segment?(point, a_point_on_polygon, trailing_point_on_polygon)
(a_point_on_polygon.y <= point.y && point.y < trailing_point_on_polygon.y) ||
(trailing_point_on_polygon.y <= point.y && point.y < a_point_on_polygon.y)
end

def ray_crosses_through_line_segment?(point, a_point_on_polygon, trailing_point_on_polygon)
(point.x < (trailing_point_on_polygon.x - a_point_on_polygon.x) * (point.y - a_point_on_polygon.y) /
(trailing_point_on_polygon.y - a_point_on_polygon.y) + a_point_on_polygon.x)
end

And here’s what I think this algorithm is doing: If you take any point and draw a ray from it, in any direction, and that ray crosses through the sides of a polygon 0 or and even number of times then the point is outside the polygon. If the ray crosses the sides of a polygon an odd number of times then it is inside said polygon. If you don’t believe me get out a pen and paper and draw it or look at this wikipedia article.

So the contains_point? method starts out by assuming the point is outside the polygon since its ray has yet to cross any of the borders. Then it needs to look at each line segment of the polygon (otherwise known as a border or edge) so, if you look at the i and j iterators, they are working their way around the polygon and are used to define a point on the polygon and the point immediately preceding it. Now the algorithm looks to see if the point in question is between the max and min y values for a given line segment using the appropriately named point_is_between_the_ys_of_the_line_segment? method. If that’s true then a horizontal ray could travel through the current line segment. But it might not, depending on which way it is traveling. So the next method ‘ray_crosses_through_line_segment?,’ checks to make sure. Now if the ray has crossed through the line segment, or edge, of the polygon you set contains_point to the opposite of whatever it was. Why? Well if it had crossed 0 times contains_point would be false and one more crossing would bring it to a total of 1 (an odd number) and so contains_point should now be true. Every time you find another crossing you flip from odd to even or vice-versa and so you must flip the variable also.

Not too hard, huh? But if there are many sides to your polygon (and we sometimes have lots of sides) then this can be quite a slow operation to do en masse (and we are planning to do it very en masse) so while doing some research for the refactor I found an optimization:

1
2
3
4
5
6
7
8
9

def outside_bounding_box?(point)
bb_point_1, bb_point_2 = bounding_box
max_x = [bb_point_1.x, bb_point_2.x].max
max_y = [bb_point_1.y, bb_point_2.y].max
min_x = [bb_point_1.x, bb_point_2.x].min
min_y = [bb_point_1.y, bb_point_2.y].min

point.x < min_x || point.x > max_x || point.y < min_y || point.y > max_y
end

A bounding box is the smallest possible box that can completely contain a polygon. If you draw a bounding box around the polygon, check to see if the point is outside the bounding box, and it is then you’re done. You fail fast because if the point is outside the bounding box there is no way it is inside the polygon. GeoRuby gives you a bounding_box method for free (but it’s not too hard to find the max and min x’s and y’s of a polygon) so I was able to create the above method and insert it into the original pretty easily:

1
2
3
4

def contains_point?(point)
return false if outside_bounding_box?(point)
# rest of method as shown above
end

Now contains_point? will give up early if there’s no chance.

metric_fu graphing, 1.9, and 'awesome' templates

Tue, 14 Jul 2009 03:12:00 +0000

So there have been a bunch of metric_fu releases since I announced 1.0.0 on this here blog. It’s now up to version 1.1.4 and we’ve added:

Flog, Flay, Reek, Roodi, and Rcov now have graphs over time (thanks Édouard Brière).
Cool, dare we say ‘Awesome,’ templates for the metrics (thanks Nick Quaranto and Edouard Brière).
Flog report now tracks average Flog score per method and the average of the worst 5% of your methods.
MetricFu now works with Ruby 1.9.1 (thanks to Robert E. Rouse for his help).

On to the pics:

Homepage
Google Group
Github Repo

Enjoy!

ORD Sessions and and introducing Flurn

Fri, 19 Jun 2009 12:35:00 +0000

I went to the first ever ORD session last night sponsored by Inventables and Google Chicago and worked on some metric_fu stuff. The idea of the ORD sessions is that there’s a lot of open source developers in the Chicago (ORD is the O’Hare airport code) area so why not get them together to hack on some stuff. I’d say over 20 people showed up and had a great time. It didn’t hurt that every one who came got an unlocked google phone as a free surprise gift. When they announced, earlier in the evening, that there would be a surprise we all thought: “Oh cool, I’ll probably get a t-shirt or a squishy ball.” So the phone sort of blew us away.

As to metric_fu, Jeff from Inventables and I paired on a new meta metric called “Flurn.” It’s a combination of FLog and chURN and it tells you about files that not only are very complex but also change all the time. This is pretty bad because if the file is in constant flux and it’s got a lot of convoluted code then that’s a bunch of bugs waiting to happen.

“Flurn” is available through the appropriately named ‘flurn’ branch of metric_fu on github if you want to check it out before it’s released in a future version of metric_fu.

Speaking of future versions of metric_fu, there should be one coming out soon with ‘awesome’ templates and graphs of metrics over time. Stay tuned.

Tribune Article about the Craftsman Swap

Mon, 15 Jun 2009 12:33:00 +0000

The Chicago Tribune wrote up an article about the craftsman swap I participated in 2 months ago:
http://www.chicagotribune.com/business/chi-mon-craftsman-swap-0615-jun15,0,1513297.story

My thoughts on it are here:
http://jakescruggs.blogspot.com/search/label/craftsman_swap

Basically Obtiva and 8th Light exchanged developers for a week. If you want to know more (and why) click on the links above.

Iteration Zero

Mon, 08 Jun 2009 02:01:00 +0000

I spent the week doing an iteration zero for new project. The idea behind iteration zero is to get the development environment as automated as possible so developers can spend their time coding when the real development begins.

So here’s what we did:

1. Set up new git repository

We use GitHub so that only took a few minutes.

2. Make new Rails app

Again, just a few minutes

3. Set up geminstaller and basic gems (HAML etc.)

Geminstaller installs gems needed by the project and makes sure they are there before starting the app. This functionality has been pulled into modern versions of Rails now, but I’ve found it a little wonky and I’m working with a former Pivotal Labs guy who just loves the Pivotal stuff (don’t even get him started on Tracker vs Mingle).

4. Set up developer tests.

We looked into Shoulda, but ultimately went with RSpec (although we may use the Shoulda macros to test ActiveRecord) because most of the criticisms of RSpec are that it’s too heavy. To that I say: “Give Shoulda some time and success and then see if it’s still light.” Feel free to flame my comments section.

5. Set up Selenium (Integration) Tests.

We used Polonium. We could have gone with WebRat, but no one on the project had ever used WebRat and one guy had extensive experience with Polonium. Our build now fires up a browser and clicks through the app (we will be taking pains to keep this suite short – just a smoke test really).

6. Set up Continuous Integration

We went with CruiseControl.rb. Note: Use the ThoughtWorks version on GitHub. If you go to the main CruiseControl.rb page it will direct you to an old version that doesn’t support Git.

7. Set up scripted deploy

Looked into Vlad the Deployer, but it’s lack of documentation did it in. I know it’s supposed to be much simpler than Capistrano, but we couldn’t get it to work. Capistrano is fine, but it’s basic documentation starts talking about spinners and spawners at some point. Wasn’t that like 4 years ago? Here’s my advice: Follow the capify.org “from the beginning tutorial” but skip the spinners and spawners stuff and just put this in your capfile:

namespace :deploy do
desc “Start the server”
task :start, :roles => :app do
run “mongrel_rails start -e production -c #{current_path} -d -p 3000”
end

desc “Stop the server”
task :stop, :roles => :app do
run “mongrel_rails stop -c #{current_path}”
end

desc “Restart the server”
task :restart, :roles => :app do
stop
start
end
end

That’s good enough to get you going. Basically you overwrite the stop, start, and restart tasks that cap uses under the covers. I should point out that I’m using Capistrano version 2.5.5 because all the doc for cap I find on the web never mentions the version they are talking about and it drives me crazy. Cap has changed A LOT over the years and something that was perfectly good in 2006-7 may not be so relevant now. Also, we’re using capistrano-ext version 1.2.1 so we can deploy to multiple environments.

8. Set up Demo and QA environments

After the previous task, this was easy. All we have to do is ‘cap demo deploy:setup’ and ‘cap demo deploy’ to get the demo environment up and running. QA was the same except for, uh, using the letters ‘qa’ instead of ‘demo.’ Now we can deploy with one command – w00t.

9. Set up NewRelic

NewRelic lets you profile your Rails app with ease. I’ve used it before on apps I’ve written and have found it immensely useful in tracking down slow parts of the project.

10. Make Cruise Control run a Metrics Build every 24 hours.

Yep, I really do use metric_fu on my projects. I like installing it from the get-go so we can keep an eye on things and track changes to the app.

Well, that ended up being about a week for a pair of developers (with some interruptions for meetings and other stuff) and a pretty good iteration zero if I do say so myself.

Hey did you know that some versions of Ruby 1.8.6 don't return the right error codes?

Thu, 04 Jun 2009 03:07:00 +0000

I didn’t. The box I was setting up CruiseControl.rb on today had some crazy early version of Ruby 1.8.6 and so test failures returned a code of ‘0’. Which meant the build would not fail until I upgraded to a more modern 1.8.6. Patch level 114 did the trick. Why are we using 1.8.6 on a new project? Well mostly so we can run along side an older Rails app using 1.8.6, but also because 1.8.7 doesn’t really offer that much more to justify the headache of running two rubys on one box, and finnaly because I have no idea when 1.9x will be acceptable on all the gems I use.

Good times.

Finding a Rails Route from a Path

Mon, 01 Jun 2009 01:39:00 +0000

Fred Polgardy and I were looking up how to find out what path in the browser corresponds to what controller/action combination in a Rails application today and I realized that I’ve looked this up before… So I’m putting this information here so I can find it in the future:

(from inside the console)

> ActionController::Routing::Routes.recognize_path ‘/seo/hotness/path/705105’

=> {:controller=>”products”, :action=>”show”, :id=>”705105”}

Yeah I know you can do a ‘rake routes’ and hunt around but this gets me where I need to be quicker.

Delete Like a Crazed Madman

Thu, 28 May 2009 00:32:00 +0000

Lately I’ve been working with this guy Felix who likes to rip the application into shreds when confronted with a problem. Initially, it’s a bit scary. I know we have source control and we can get it all back, but ripping out large crucial swaths of code just seems so violent and yet it’s a pretty effective strategy.

On this one project we’ve had a problem for well over a year of our tests not rolling back the database after each test. We’ve got all the configuration set up for that to happen, but it just doesn’t and nobody could figure out why. One of the problems is that this is a Rails application that is 2.5ish years old and serves millions of users every day. So in order to make that happen back in the early days of Rails we had to extend and modify a lot of the Rails framework. Apparently one of those modification broke the mechanism for rolling back the database after every test.

Not rolling back the database means that one tests creation/updating of a model can break other tests so we have to remember to clean up after ourselves. Invariably someone forgets to and we get a lot of weird test failures that have to be worked out (or worse disabled with the comment “runs by itself but not with all the other tests”).

Felix’s solution to our tricky problem was to create a new crazy simple ActiveRecord model ‘Foo’ that one had one validation: Its name had to be unique. Then we wrote a crazy simple set of tests that tried to create and save a Foo object in separate tests with the same name. If the database was rolling back properly then there should be no problem. Of course they failed in our messed up environment. Now since this model Foo has no dependencies on any of our complicated models, we could delete lots of stuff without breaking it. And we did. We deleted every plugin and gem we had until we found the problem. Of course we did it pseudo-binary search style – cutting out large sections of gems/plugins in one go to see it it had any effect and then narrowing it down when it worked. In a relatively short amount of time we had found the culprit: A plugin we had written that attempted to make Rails rollback multiple databases in nested transactions. Now the best solution would be to move up to a more modern version of Rails where this just works (or at least the internet says it does), but our solution was to make this particular plugin not load while in the testing environment. Not a perfect solution, but a good enough one.

The real take away from this exercise for me is that simplification and deletion are powerful tools in bug fixing toolkit.

Git isn't Chatty

Mon, 25 May 2009 23:39:00 +0000

Recently we deployed to production without a key bugfix and my assumptions about git were to blame.

We, like other people who use git, do a lot of our bug fixing in a branch and then merge it into master when it’s time to release. So here’s the procedure for the guy who’s turn it is to do a deploy (which was me):

Do a ‘git pull origin master’ in the master branch
Get a local tracking branch of the bug-fixes branch
Merge the bug fix branch into master
Tag the master branch with the name of the release (our deployment keys on tags)
Start the automated deploy script

Easy right? Except that my years with subversion have put in my brain this assumption that my source control will talk to the remote server all the time. But git’s model is to use all your local info unless you specifically tell it to go over the wire. So when I told git to create a local tracking branch it did using the latest information it had locally about that branch. The last time I had done a general ‘git pull’ was a day ago so I got a version of the bug fix branch that was missing a commit that had happened the day of the deployment. And of course when I merged into master I was missing a crucial commit.

One of the cool things about git is the fact that you can do a lot of development without an internet connect, but I’ve found that a fair amount of the gotchas I’ve run into with git trace back to my expectation that it will talk to the remote server when it’s preference is to stay local.

iMeow

Mon, 11 May 2009 23:26:00 +0000

So my friend Leah has lots of ideas. And one of them was that people would pay money for a screeching cat sound on their iPhone.

Everyone within earshot thought about it and the consensus was that, yeah, I might pay a dollar for that. So Leah and I set about working on the iMeow application but the iPhone development one-two punch of dragging/clicking and Objective-C soon drove us to despair. I, the lessor developer, gave up but she stuck with it (and pulled in some other Obtivians with iPhone knowledge such as Turner, Nate, and others who I’m probably forgetting) so I’m proud to present iMeow (iTunes link).

It’s a silly little good time.

Rails Conf 2009 Third Day of Sessions

Thu, 07 May 2009 20:35:00 +0000

Last night I chose quantity over quality and ate at a Brazilian steak house. Good times.

Webrat: Rails Acceptance Testing Evolved Bryan Helmkamp (weplay)

Bryan started out his presentation by handing out Tylenol for those who went out last night and Budwiser for those who didn’t. The crowd was appreciative.

Bryan show some typical Rails integration testing (simulating clicking through the app without opening up the browser) and compared it to the same test in Webrat. The difference between the integration test and the webrat enabled test was pretty dramatic. The Webrat test was much cleaner and easier to read.

Webrat parses the returned html with Nokogiri (which he highly recommends) and so can do some pretty interesting stuff. click_link, fill_in, check (and uncheck), choose, select, attach_file, and click_button all have a simple syntax.

Webrat automatically is checking for the success of the request so you don’t have to assert success everywhere. It also verifies that form fields are there when you try to fill them in. Nice.

There is a Selenium adapter for Webrat so that you can flip your tests over from simulating the browser to actually firing up a browser and clicking through.

good talk.

When to Tell Your Kids About Client Caching Matthew Deiters (Inc.)

This was another talk that had a lot of good content but went pretty fast, so the following are excerpts I managed get into TextMate:

Why should you care about client (browser) caching? Well, 80% of the total time to pull up a webpage is spent on the wire or the client side.

Entity Tags: When configured, the server sends an E-Tag to client (browser). Then the next time they look for it the browser sends the E-tag back and if the server determines nothing has changed then it send the super small “304 Not Modified” and browser re-renders the page super fast.

Rails can calculate the hash of a page and compare the last time sent to the current and if there is no change then “304 Not Modified”

Don’t use etags on static content – you can set up your apache config to not look again for 10 years.

use commit times for the cache buster image.jpg?3123123 <- not just the time but use the commit time of the file

To stop ajax from being cached (which can cause problems): (try not to use method post – it generates 2 requests) Use HTTP-Headers => http://github.com/dancroak/no_cache helps you do this

Less requests are better so combine/minifi/compress your javascript and css files for fewer asset requests. There’s a lot of tools to do this.

bundle_fu (http://code.google.com/p/bundle-fu/) is one of them.

Let Google host your javascript. Google hosts most popular javascript frameworks so you can rely on them to serve up these assets for you. include_google_js does this for you (http://github.com/chriswarren/include_google_js)

Automated Code Quality Checking In Ruby And Rails Marty Andrews (Cogent Consulting Pty Ltd)

This was the other code metrics talk at Rails Conf. Turns out Marty Andrews is also a former ThoughtWorker. He was (and still is) in Australia during my time at ThoughtWorks so we just met at Rails Conf. Nice guy.

There was a fair amount of overlap between our talks so I’ll limit my comments to things that were not mentioned in my talk.

Static analysis is when you analyze the code without running it. No app runtime required.

Feedback for your developers is key to continual improvement.

Marty is a big fan of failing the build on poor Flog, Flay, and Roodi results. I’ve resisted putting that functionality into metric_fu in the past because I wanted metric_fu to be more about collecting data than judgment. But maybe I was wrong. I opened up a new thread for discussion of this on the metric_fu Google group (http://groups.google.com/group/metric_fu). If you have an opinion on this please comment on the thread.

He showed a bunch of examples of poor Flog/Flay/Reek/Roodi results from the Rails source code – Not to bash on Rails but to say that even awesome developers we all respect can check in some problematic stuff. We could all use some watching over.

Marty also has a tendency to call “metric_fu” “metrics_fu.” Which is funny to me.

All right then, I gotta go catch a flight so this is where Rails Conf 2009 ends for me. Hope you enjoyed these write-ups.

Rails Conf 2009 Second Day of Sessions (Evening)

Thu, 07 May 2009 15:21:00 +0000

If you’re looking for the slides from my talk, you can find them (and everyone else’s) here:
http://en.oreilly.com/rails2009/public/schedule/proceedings

I feel like I must confess to you, faithful blog readers, that last night I ate a $240 steak at CraftSteak. But I do not apologize – It was worth it.

Before I give anyone at Obtiva a heart attack I should point out that I will not be expensing that meal.

Also, have you seen Toby Tripp’s Meeting Ticker? (http://tobytripp.github.com/meeting-ticker/) You enter your billable hourly rate, the number of people in the meeting, and press start. It then counts up the amount of money wasted, er, spent. Useful for those clients who are addicted to meetings.

Now on to the keynote:

What Killed Smalltalk Could Kill Ruby Too Robert Martin (Object Mentor Inc.)

First of all it’s probably worth tracking down the video of this talk and watching it. Uncle Bob is a crazy talented speaker who knows way too much about software.

What was it that killed Smalltalk? Kent Beck – “It was just too easy to make a mess”
Also it was written on top of a database (All the code went in there).
And Java was free.
But mostly the mess.

Boy scout rule: Leave the campground cleaner than when you found it.

Perform a random act of kindness on your code every time your check it out.

TDD can be a solution to the mess. (I would also add that code metrics are another.)

The tests allow you to clean the code.

Professionalism: The disciplined handling of power.
Just because you can do a thing does not mean you should do a thing.

What is it that might save rails:
Discipline (TDD)
Humility (Try not to make everyone hate us by snickering too much about other languages)
Acceptance of solving the dirty problems. (ugly database schemas in the enterprise)

If you’re tempted to ignore these warnings – Remember the fate of the most powerful and influential language of the 70s and 80s.

Rails Conf 2009 Second Day of Sessions (Afternoon)

Thu, 07 May 2009 00:35:00 +0000

My talk (Using MetricFu to make your Rails code better) went well. I’d say there were 200ish people in the crowd and they all seemed to enjoy it. There were a bunch of questions and some verbal commitments to contribute to MetricFu.

Rails in the Large: How We’re Developing the Largest (Enterprise) Rails Project in the World Neal Ford (ThoughtWorks), Paul Gross

Full disclosure: Paul and Neal are friends of mine from my ThoughtWorks days.

The talked about the trials and tribulations of developing a very large Rails project over a few years. The project is called Online Vehicle Exchange (OVE) and it’s a sort of EBay for car dealers (it isn’t open to the public).

The have 11 dev pairs (22 devs). And started out with mac minis but later switched to mac pros – they probably should have switched over earlier.

Testing:
Fully disconnected unit tests. They use UnitRecord (which reads your schema.rb file to). If you accidentally hit the db then UnitRecord fails the test

They run 9000 unit tests in 41 seconds.

The run 4000 functional tests in 248 seconds. There is no mocking the db in functional tests. Now tests that hit the db are slow so they had to invent some strategies for

They use Deep Test (open sourced) to parallelize your test suites. 8 cores on a mac pro – so 8 parallel test runners. They also used Distributed Deep test (also open sourced) - to run the tests on another computer. They had 3 linux boxes in the corner that just ran tests from developer mac minis.

Selenium grid (more open source) was used to spilt selenium across multiple cores and machines. As many boxes as you want to parallelize out the tests.

Always fight the battle to keep tests fast.

They used a projector to throw Mingle up on the wall so they can keep an eye on the cards and the progress of the iteration.

They wrote cc_board – an open source project that aggregates multiple cc.rb instances to give an overall view.

jukebox.rb – plays a song when the build breaks and random music other times.

No email installed on the pairing stations – giant time sink.

Software is more about communication than technology.
Co-location Rocks (war room style programming)
Pairing is vital to this large project so that everyone has some idea of what’s going on.

One click deployment to any environment. Rake commit to check in (runs all tests then commits for you).

They used radmind to keep all of their mac’s software identical to each other.

Monkey patches for the project live in a specific folder and are modularized.

It took 1 pair 6 weeks to upgrade from Rails 1.2.3 to 2.2 – Large projects can be tough to upgrade.

Interfaces are Dumb (and that’s a Very Good Thing) Erik Kastner (Wine Library / Meta

ateM)

Electricity – has a dead simple interface. Plug it in and it works.

Your APIs should be the same way.

Why expose an API? You are not as smart as the collective intelligence of all your users.

Twitter and Flickr both have really easy to use APIs. It’s probably responsible for some of their explosive growth.

GitHub has a webhook that will post every time someone pushes to a repo.

Keep in mind that the map is not the territory. A developers mental map often does not match the users mental map. This leads to API confusion. Make pains to keep your API simple and solicit lots of outside feedback when people start using it.

Time for Uncle Bob’s Keynote now…

Rails Conf 2009 Second Day of Sessions (Morning)

Wed, 06 May 2009 19:11:00 +0000

Sponsor Keynote: Agility in Deployment - Rails in the Cloud Jon Crosby (Engine Yard)

During the morning session Engine Yard announced their new Rails on EC2 (Amazon’s compute cloud) solution: Flex

You can one button deploy to the cloud. It is a self healing cluster: Each node is in a load balancing pool. If one fails they automatically startup a new one to take over.

It can use a web interface or capistrano.

You can create a clone of the whole production environment with the data. So you can practice deploying to your cluster. Then you can blow away the staging environment when you’re done with it. Nice. Keeping a staging environment around all the time is very costly if you want it to accurately simulate your production environment.

Flex will be available in June

Chad Fowler then introduced Chris Wanstrath for his keynote. While introducing him he mentioned that when he first saw git he didn’t like it much. But he, and a lot of others, adopted it because of Github. I agree.

Chris Wanstrath (GitHub) Keynote

Some of his thoughts:

It’s not too hard to become a famous Rails Blogger/Speaker, but the people he respects are the coders who contribute year after year and blog as an afterthought (or even not at all).

Do you want to be Kid Rock or a good programmer?

The inspiration for GitHub was all the friction in the open source project community. GitHub is all about making open source and collaboration easy. You can start an open source project in a few minutes instead of applying to SourceForge which takes days.

Time to go give my talk…

Rails Conf 2009 First Day of Sessions (Evening)

Wed, 06 May 2009 18:57:00 +0000

Ruby Heroes Award Ceremony Gregg Pollack (Rails Envy)

Gregg gave out the Ruby Heroes Awards:

Brian Helmkamp for Webrat
Aman Gupta for EventMachine and AMPQ
Luis Lavena for One-click Ruby Installer (for Windows)
Pat Allen for Rails Camp and thinking_sphinx
Dan Kubb for DataMapper
John Nunemaker for HTTParty and RailsTips (the blog)

Keynote Timothy Ferriss (The 4-hour Workweek)

The keynote was run as a conversation between David Heinemeier Hansson and Tim Ferriss.

Tim worked from 7am to 9pm when he realized that his income had no meaning. He took some time off and during that time he limited his work to 2 hours every Monday of checking his email. Profits went up 30%. Turns he was way over managing everything.

I’ve read the 4-hour work week and it’s a fun and wild ride. The title is a bit of an eye-grabbing misnomer. It’s more about viewing time as valuable commodity.

Tim practices an information diet. There’s way more information in this digital world than you can possibly consume. He limits his consumption severely and it hasn’t affected his performance at his job.

“You want to have rituals and conventions for everything that is not part of your core competencies.”

The sound was a little low, so it was kinda hard to hear and I think that made the crowd a little restless. Oh well, I’m glad they keep trying to bring in outside people every year.

If you’re a Tim Ferriss hater, here’s an funny rant against him and his practices:
http://blog.penelopetrunk.com/2009/01/08/5-time-management-tricks-i-learned-from-years-of-hating-tim-ferriss/

Rails Conf 2009 First Day of Sessions (Afternoon)

Tue, 05 May 2009 23:11:00 +0000

Did you know the Las Vegas Hilton charges you 3 dollars to print out a boarding pass? Jerks.

JavaScript Testing in Rails: Fast, Headless, In-Browser. Pick Any Three. Larry Karnowski

So blue_ridge is a javascript testing tool that mashes up a bunch of other tools to make testing javascript more or less like writing RSpec test. It’s pretty cool. It uses Screw.Unit for the RSpec like lingo, Smoke for mocking, and Rhino to run the javascript tests without a browser (so you can integrate it into your Cruise Control build).

http://github.com/relevance/blue-ridge/tree/master

Blue-ridge is JQuery opinionated but can use Prototype without too much trouble.

It has a textmate bundle so you can run your javascript tests with command-r

You can have multiple describe blocks like RSpec.

Definitely worth a look.

Smacking Git Around - Advanced Git Tricks with Scott Chacon (GitHub)

Scott is very smart and really funny but he talked very fast so if you want to see the full slide deck you can go to:

http://www.tinyurl.com/gitrailsconf09
or here for the cheat sheet:
http://www.tinyurl.com/gitrailsconf09-cheat

Here’s some selected tips I was able to jot down:

You can use partial sha1’s to look up commits.

master^1 is parent of master
master~2 is two commits ago on master

Ranges:
[old]..[new]
ce04.. Means everything since then

git log orgin/master.. What am I going to push?
git log master ^orign/master Same as above

git log master bacon ^origin/master What is in the ‘bacon’ and ‘master’ branches that is not in origin master.

git log –graph Produces an ascii graph
gitk Produces visual graphs and takes pretty much the same arguments things as git log

git diff topic Means how do I make topic look like head – probably not what you want

git diff HEAD…topic What happens if I merge in the topic branch – useful

No rebasing on any commit you’ve pushed upstream! It messes everything up.

Subtree merging is an alternate to submodules. Tim Dysinger has a write-up on how to use it. Find it at: http://www.tinyurl.com/braidgit

git blame -C a_file.rb If a line was moved from another file then give me the last time it was changed from another file. Cool. You can even adjust how hard it looks for that line in another file.

git bisect is a binary search where a bug was introduced

git bisect start
git bisect bad The current commit is bad
git bisect good ae03efg… The last time you know it worked

Bisect will now take the range and check out the middle one, you run the code to see if the bug is there. If it still is:
git bisect bad And you get a new commit

if the bug isn’t there:
git bisect good And you get a new commit

It keeps doing a binary search of commits (using your feedback) until you narrow it down to exactly the commit that caused the problem. Neat.

Then ‘git bisect reset’ to get back to normal.

git config –global color.ui auto For Colors

There’s a way to set up git to diff word docs. Which is cool. See the link to the slides to see how (it went by rather fast).

Excellent talk.

Rails Conf 2009 First Day of Sessions (Morning)

Tue, 05 May 2009 22:48:00 +0000

Quick note: What’s up with all the moths in this convention hall? They keep wandering over to see what I’m typing. Odd.

David Heinemeier Hansson’s Keynote:

Here are some thoughts expressed by David during his keynote:
You can win argument merely based on the strength of your arguments, most often it just takes some time for good ideas to sink in.

7 Reasons I switched back to php (Derek Silvers) and the Twitter Fail whale (Twitter’s scaling problems with Rails) were pretty big issues – for about 2 days on Digg/slashdot. His strategy with this is to relax, read the credible info and learn. Criticisms about Rails scaling will never go away despite all the high profile successes.

Merb was the most credible internal (from inside the community) competitor/critic. And now they are merging in Rails 3.

Philosophy of Rails 3:
Lock up the unicorns… huh? You can’t get every feature you want and still release before the end of the world.

No sacred cows. If you want to break every Rails app out there with your idea: it’s still on the table. You have to have an overwhelming argument to get it accepted, but they will listen.

Rails is still committed to getting everything by default. But now you can have less if you need it.

Progress of Rails 3:
New Router:
Faster
route by subdomains, user agents, etc.
route to rack machinery

Cross site scripting protection:
Previously not automatically escaped.
The convention will be reversed

<%= comment.body %> escaped
<%= raw comment.body %> not escaped
The rails helpers will automatically escape
You can mark a sting as html safe Ex. “I’m safe!”.html_safe

Javascript - unobtrusive and agnostic
Not so much included javascript in the template, but will pull it out into a file. The javascript that Rails used to put on the page when your did, say, link_to_remote will now be abstracted into a file that looks for ‘data-remote’ attribute on the anchor tag.

More Agnosticism
Action ORM (so you can use form_for a data mapper object)
Generators are now more open to plugging in stuff

The great Refactoring
Cleaning up a bunch of internals to make them less cluttered and speedier.

The real secret of high productivity:
Renegotiate requirements. If you look at requirements as set in stone you will leave a lot of productivity on the floor. Most of the time the business will take an 80 or even 60 percent solution if it takes a lot less time. Resist gold-plating.
Are you a programmer or a partner?

The GitHub Panel with Chris Wanstrath (GitHub), Tom Preston-Werner (GitHub), PJ Hyett (GitHub), Scott Chacon (GitHub), Jon Maddox (Mustache, Inc.):

The entire GitHub team answered questions from the internet and the audience. Here’s some of the highlights:

They open source grit, but not the whole application. Because they want to make money and they want it to be their own vision. Maybe someday in the future.

One of the devs said he wished they had never built the rubygems system. It takes a ton of work and it doesn’t make any money. Chris Wanstrath piped up that it’s an effort of goodwill. I would say so. I’ve never got around to putting the MetricFu gem on Rubyforge and rely solely on GitHub to provide my gems. Maybe I should change that.

They are planning to introduce more sociableness like groups and customizing your feed with weighting. Which would be very cool.

They are profitable and are able to pay themselves nice salaries. This was not the case 6 months ago. Chris Wanstrath said that he spent 6 months living on savings and writing code every day. It was the best time of his life.

They have no office. But they do have a cafe they hang out at a lot.

Off the cuff comment when asked about crashing firefox: “The site is optimized for Safari.” Ouch. Before you get bent out of shape – they were joking. They plan to re-do the project page to better deal with lots of branches or tags.

The network data (used to make the network graph) is all available from the api.

They use 9 slices on Engine Yard.

Discussion Panel: Women in Rails with Desi McAdam (Hashrocket Inc and DevChix Inc), Sarah Mei (LookSmart), Lori Olson (Dragon Sharp Consulting):

Desi declared that they would only spend 5 minutes on the Couch DB Pr0n scandal. They wanted to spend most of the time talking about getting more women into the community.

One way is to create groups for women. Example: DevChix (http://www.devchix.com/)

Also being more visible so that women know you’re out there. One of the panelist said that she feels like that’s hard for a lot of women but that it’s important to have a blog or twitter or whatever.

There’s a lot of women who drop out when they have kids. And when they try to come back there’s a gap in their resume’ so they can’t even get an interview.

You don’t see a lot women in the Rails community (vs Java) because of the lack of maternity leave in small companies (which tend to adopt Rails more easily). The cutting edge languages and companies tend to lose out on women candidates through their lack of benefits.

In the 80’s 25-30% of CS students were women. It’s since gone way down. During the tech boom most CS programs made their first 2 classes super hard to weed out people as everyone was trying to be a programmer. This had the effect of driving female enrollment way down.

Desi talked about the need for sponsorship of women going to Rails conferences. Her organization, DevChix, has sent people to Rails Conf at reduced prices. Child care is another issue.

There are a lot of women who are developers in javascript/flash and call themselves designers. But they are really developers who just get paid a lot less.

Even women who develop often see another women at a conference and tend to assume she is a vendor salesperson.

One of the panelists said that she thought Matt Aimonetti (of the Couch DB pr0n scandal) made a mistake in just not thinking about the female developer community. Another said that you can’t argue with someone about how they feel. Desi pointed out that had Matt just apologized (and not said I’m sorry that you were offended). And also that David’s (DHH) tweet (“Do you think porn is an appropriate metaphor to be used in business presentations?” – Absolutely. Especially in jest.) did not help.

Tutorial Day at Rails Conf 2009

Mon, 04 May 2009 23:13:00 +0000

First off, I’m going spend a minute in non-technical land and say that my wife and I had a fabulous time in Vegas over the weekend. She got to be part of a magic trick at the Penn and Teller show and won 50 bucks from a Wizard of Oz slot machine. Here’s some pics of our trip in case you want to see: http://www.flickr.com/photos/7589554@N02/sets/72157617613438444/

The first tutorial I attended was “Running the Show: Configuration Management with Chef” by Edd Dumbill. Poor Edd had a demo involving talking to machines across a network and it didn’t go well early. Live coding, combined with installing through the internet, and setting up a local intranet was sort of a triple threat. Any one of those is hard to pull off in front of a few hundred people and all three proved too much. We took a break an hour and a half in and the network issues were mostly worked out.

Chef seems to be sort of a very intense Capistrano. What if you wanted to deploy your app to a bare unbuntu install? Or any other bare install. Well Chef can help you write recipes to set up Apache (and it’s config files), Passenger, and any gems you need. The machines you set up (called nodes) will call back in to the Chef server periodically to see if you’ve changed the configuration. If you have they will update themselves automatically.

Next up was “Solving the Riddle of Search: Using Sphinx with Rails” by Pat Allan. Pat started out with a brief overview of Sphinx (a text search engine) and thinking_sphinx (the plugin/gem that integrates sphinx with Rails). Then he showed a little bit of queries and indexing logic. And then the tutorial sort of ground to a halt as everyone tried to install sphinx (which needs to be built from source via make for OSX). There had to be at least an hour of Pat wandering around trying to help people install it. Which was a little frustrating to the people who had read the email from a week ago and had already installed sphinx. Yes, I was one of those frustrated nerds. Anyway, it gave me some time to work on my slides.

And they needed some work! MetricFu has changed a lot since I last gave a talk about it at Ruby Conf. Since then we’ve integrated Flay, Reek, Roodi, and re-written almost the whole gem to support yaml serialization. The metric_fu community has been busy.

Whoops, Pat has started up the tutorial again. We’re using a project located at: http://github.com/freelancing-god/sphinx-tute/tree/master to try out sphinx live.

Good news, my Sphinx install works!

Sphinx does delta index updates – where you update only the changes.

> Place.create :name => “Whatever”
> Place.search “Blank”
=> [Place id: 484, name: “Whatever”, address: nil, city: nil, state: nil, country: nil, lat: nil, lng: nil, description: nil, created_at: “2009-05-04 22:43:15”, updated_at: “2009-05-04 22:43:15”, delta: true]

See how the delta column is true – that guy needs to be updated.

Geo-Searching for models that have Latitude and Longitude is pretty cool:

@places = Place.search :geo => [0.630713104998402, -2.00979852231013],
:order => “@geodist ASC”

@places.each_with_geodist do |result, distance|
#distance is in meters
end

And then Pat said he was done with the tutorial about an hour early. He seems like a nice guy and all but there just wasn’t a lot of content for a 3.5 hour session.

To sum up today’s tutorials I’d say that I’m happy that, as a speaker, I didn’t have to pay for them.

MetricFu 1.0.0 is Out

Thu, 30 Apr 2009 16:59:00 +0000

What’s new:

Merged in Grant McInnes’ work on creating yaml output for all metrics to aid harvesting by other tools
Supporting Flog 2.1.0
Supporting Reek 1.0.0
Removed dependency on Rails Env for 3.months.ago (for churn report), now using chronic gem (“3 months ago”).
Almost all code is out of Rakefiles now and so is more easily testable
Metrics inherit from a refactored Generator now. New metrics generators just have to implement “emit”, “analyze”, “to_h” and inherit from Generator. They also must have a template. See the flay generator and template for a simple implementation.
You now define the metrics you wish to run in the configuration and then run “metrics:all”. No other metrics task is exposed by default.

Full documentation, as always, is at: http://metric-fu.rubyforge.org/

Let the Google group (http://groups.google.com/group/metric_fu) know about any bugs you find.

Women and Code

Thu, 30 Apr 2009 02:05:00 +0000

So I’ve been kind of heads down trying to get metric_fu ready for Rails Conf, so I had only vaguely heard about the recent controversy over Matt Aimonetti’s recent presentation “CouchDB: Perform like a pr0n star.” When someone did mention it to me I thought of David Heinemeier Hansson’s famous “F— You” slide or Zed Shaw’s rants or any given bit of profanity/edginess at Rails conferences and shrugged it off as a bit of a bad idea.

Earlier today we were having our weekly team meeting when the topic of the controversy came up and my friend Leah was quite upset by it. And I realized that she has to deal with a lot of crap everyday being the only female coder in our office, at user groups, in conferences. We’ve had experiences where clients treated her in ways they would never have treated a man. And that sucks.

I’ve read Matt’s post about the experience and I believe he didn’t mean any harm, but I don’t think you can show pictures of mostly naked ladies in a presentation when the female developer community is such a visibly discriminated against minority.

Announcing the Metric Fu Google Group

Tue, 28 Apr 2009 23:37:00 +0000

I really should have done this a long time ago, but you know how it goes.

Anyway, if you’re using metric_fu and have a question or want to discuss it’s direction/development you can now here:

http://groups.google.com/group/metric_fu

There’s some big changes on the horizon for metric_fu – If you want to get a preview, head over to github and take a look at the to_yaml branch of metric_fu. I’ve been pulling in Grant McInnes changes to be able to serialize the output of all the metrics. This means that there will be a yaml file created after each run which can be used by other projects. A number of people have asked for this feature so they can do analysis of their metric data.

However, this will probably break some uses of MetricFu as it is an almost complete re-write. What’s cool about this branch (besides the output) is that almost nothing is in rakefiles anymore and so is more easily testable. If you use metric_fu and have an opinion on this, or anything else, join the group and make yourself heard.

Smells of Testing (signs your tests are bad)

Tue, 28 Apr 2009 02:54:00 +0000

I spent the weekend at Citcon (Continuous Integration and Testing Conference) in Minneapolis. Citcon (which is pronounced Kitcon – because it sounds cooler) is an open spaces conference which means that there are no scheduled talks. The people who show up propose ideas and then they vote on what they want to see. It sounds like it would never work but it actually works very well. For instance, Toby Tripp proposed a session on testing smells and it got enough votes to get a session but when I showed up there were only 6 people. Which kinda bummed me out until we started talking, then I realized that we had exactly the right 6 people and another 20, 50, or 100 half interested dudes typing away on their computers really wouldn’t have added to the talk. As it was the talk was intimate and to the point.

We came up with the following smells (a smell is a sign that something might be wrong – Fowler used this metaphor to great effect in his seminal book “Refactoring”):

Requires Supervision:
A test is not automatic – it needs user input to function. Someone has to watch the test and enter data periodically.
Solution: Don’t do that.

Refused Bequest:
A mega setup that some tests ignore, some use, and most rely on but only for a few lines. When any individual test fails it’s hard to parse the setup to see what’s going on.
Solution: In RSpec you can use nested describes/contexts to have mini setups for each situation. Or you can write private method setups that are more focused. Or you can create multiple test files for different contexts.

Many Assertions:
When a test tests way too much its failure is often hard to track down.
Solution: break the test up into smaller atomic chunks (test cases) that zero in on one or two things a method should do.

Large Test File:
Things can be hard to find in a large test file and its documentation value is lost.
Solutions: It could be the class under test is too big and needs to be broken up into smaller discrete bits. Or you may have redundant tests that should be removed. Maybe you need a lot of setup and could move to a data or factory driven test. Or you could extract testcases into separate files (or describe blocks) each with a specific purpose to organize things.

External Data:
A test that needs external data (fixtures, files, helper classes, etc.) in order to run can be hard to understand as the reader must file hop to get a clear picture.
Solution: Avoid this, if possible, by using standardized Object Mothers, mocks/stubs, and removing redundant tests.

Excess inline setup:
Too much setup in every test means the reader has to skip most of the test to get to the meat.
Solution: Extract to setup methods or different test files. Focus and organize your tests.

No Assertions:
A test that has no assertions only exists to lie about code coverage.
Solution: Don’t do that. Delete the test or add an assertion (mock expectations count as assertions).

Testing the Framework:
If you’re using a framework you pretty much have to assume it works. Testing it along with your other tests only clutters things up.
Solution: Delete the framework tests.

Test Envy:
A test that is really testing some other class than the one it is supposed to. A common example of this is testing model validations in the controller when they should be tested in the model test.
Solution: Move the test or delete it if it is redundant.

Brittle Tests:
Tests the break all the time when small things change undermine the importance of test breakage. Examples are testing that a greeting is inside an “h1” in a view or the exact wording of text on the screen.
Solution: Perhaps you don’t really need to assert that. Or it might be a sign of a deeper problem with your code when it’s tests are fragile.

Order Dependent Tests:
Tests that need other tests to run before them really screw things up when things get moved around.
Solution: Make sure your tests clean up after themselves or are reasonably isolated from the outside world.

Long Running Tests:
When the feedback loop of code, test, code gets too long it can encourage bad behavior.
Solution: Spend time reducing the time of your tests. Buy faster machines. Mock out expensive operations (such as database calls).

Mock Happy:
A test that needs 17 mocks to run can be quite fragile, resistant to change, and hard to understand.
Solution: This could be a sign that the code you are testing has too many dependencies. Or you could introduce concrete collaborators. Mock/Stub out that which you don’t have access to or is expensive, but the rest can be real models. Yes this makes your tests less unit-y, but it will make them more robust and easier to understand

Not Idempotent:
Fancy words meaning your test changes the state of something permanently and thereby introduces the possibility of order dependent tests.
Solution: Clean up after yourself.

Overly Dry Tests:
DRY (Don’t Repeat Yourself) is a good idea, but pulling out all repetition can lead to some very hard to understand tests.
Solution: Some moistness is alright if it helps the reader understand the intent of the test.

Code Run Only by Tests:
Dead code (never touched in production) is very confusing to the developer trying to understand the system. Tested dead code doubly so.
Solution: Delete the test and the code. You’ll still have it in source control if you ever need it in the future.

Commented Code in the Test:
Commented out assertions or setup may mean someone took a quick path to making the test pass.
Solution: Investigate if the code should be there. If there’s no reason for it, delete it.

I got accepted into Agile 2009

Sat, 25 Apr 2009 02:00:00 +0000

My new talk “What’s the Right Level of Testing?” got accepted for the Agile 2009 conference happening late August in Chicago. I’m pretty excited to be going – the last time I was there it was called XPAU (eXtreme Programming Agile Universe).

When Your Mocks Burst into Flames

Fri, 24 Apr 2009 12:45:00 +0000

So yesterday Leah and I were working on SudokuScanner -- a cool little site that will, when it’s released, allow you to upload images of Sudoku boards and get a solution. All of a sudden, our mocks and stubs went totally batshit. One minute she and I were happily writing some tests, making them pass, refactoring, checking in, etc… and then, we got a bunch of failures. OK, no problem – we must have busted something with our latest change. The failure was a mock expectations error: UserMailer expected to receive a call to deliver_activation but did not receive it. So we went over to the failing spec and… There was no expectation on UserMailer anywhere in the spec.

That’s bad.

Ran the spec by itself and it passes.

Yep - not so good.

Reverse the order that the specs run in (spec –reverse) and a different number of specs fail, but still with the same expectation failure. And of course these new specs all pass on their own. And have no reference to UserMailer anywhere in them. I searched the project and only one spec has one expectation on UserMailer.deliver_activation, but now I have 10 failing specs that are all failing because of that expectation.

So it appears the mocking framework isn’t cleaning up after itself – Yipes!

After a bunch of exploring/messing about I found myself in the spec_helper.rb file and I noticed something weird: The line ‘config.mock_with :mocha’ that should not be commented out is commented out. Which is weird because we’re using Mocha as our mocking framework. So I uncomment it and run all the specs through rake. A billion errors all saying some variation of “method stub! does not exist” or “method should_receive does not exist.”

Got it.

So it seems we are using RSpec’s built in mocking framework on this project, but somebody started using Mocha too. Full disclosure: That somebody is probably me. I’ve been bouncing between a bunch of projects lately and some of them use Mocha and some use RSpec’s mocking framework. I probably task switched over and started using Mocha without thinking about it and it just worked. I have Mocha installed as a gem on my machine and maybe that’s enough, or maybe one of our plugins or gems requires it, but it’s there apparently in the load path so everything was fine for awhile. Of course mocking frameworks do some pretty intense meta-craziness under the covers so it was bound to happen that using both would lead to one screwing up the other’s functionality like, say, breaking Mocha’s ability to clean up after itself. Which lead to our pile of nutty failures.

Luckily changing over to from Mocha to RSpec’s mocking framework is not hard so we were able to purge the mocha syntax from the project and all was made well.

I could write a test that looks at all of my test files for Mocha syntax… Or is there a way to un-require Mocha explicitly so this doesn’t happen again?

Fifth Day of the Craftsman Swap

Mon, 20 Apr 2009 14:31:00 +0000

Day Five of the Craftsmanship Swap:

Last day!

Today started off with some more work on a soon to be released 8th Light project that will hopefully make them some money. Colin and I had a good time blowing up the app and making sure a notification email gets sent out using Exception Notifier. Later on, at a retrospective, Paul asked me if I thought it would make any money and I said that I had no idea but it was worth it even if all they got out of it was some greenfield experience for Eric and Colin. Oftentimes when you’re a new developer you get told what to do. But this project, although managed by Micah, has mostly been programmed by Eric and Colin. Which is awesome. It’s a real app that will go into production and charge credit cards, but it’s small and simple enough that they could write every line. There’s a lot of 10 year veterans of the industry who’ve never had that sort of start to finish experience and I think it’s cool they’re getting it.

In the late morning/afternoon I worked on Fresnel -- a Ruby/Limelight desktop client for Lighthouse. The idea is to fix some of the ui issues but also to enable an offline mode which would sync up to the server when connected to the internet (this feature has not yet been implemented).

And now some photos of 8th Light:

The exterior.

The lake right next door.

From left: Paul, Eric, and Colin

Lonely Colin early in the morning in the war room.

Long shot over the stairs into the meeting area (where they host user’s groups).

Where people sit in the meeting area mentioned above.

Thanks again to all involved at 8th Light and Obtiva for letting me have this wonderful opportunity to see how another small consultancy works toward software craftsmanship.

Fourth Day of the Craftsman Swap

Thu, 16 Apr 2009 22:06:00 +0000

Day Four of the Craftsmanship Swap:

Today started with Eric Smith and I working an iPhone version of Conway’s Game of Life trying to be as test first as possible. It was… Painful. I’ve been spoiled by my time with Ruby into expecting first class testing/mocking/stubbing. Even working with a guy who’s given talks on how to test drive iPhone apps there was still a lot of wasted time scratching our heads, hand writing mocks/stubs, and f-ing around with clicking and dragging lines from one box to another.

We got interrupted mid-morning by some production issues in a large Rails application 8th Light maintains/ develops. We had to write a script to find some bad data and fix it so I got to use one of my favorite tricks to mark the progress of a long running process:

10.times do
sleep 1
print “.”
STDOUT.flush
end

If you don’t put in the “STDOUT.flush” all the dots come out altogether at the end, but with the flush you get a nice dot to tell you every time you’ve completed one loop. Copy and paste the dots into TextMate and check the column number to know how many you’ve done for extra long processes.

After we straightened out the productions problems we had to go back to iPhone development. Sigh. We spent about 3-4 hours working on this application and we accomplished just about nothing. We were trying to get a button to repeatedly step forward the game using an NSTimer, but by the end of the day we were still having no luck. We did get the button to stop crashing the app altogether, so I guess that’s something.

Btw, Eric’s personal iPhone has a face plate that is 40% shattered.

Third Day of the Craftsman Swap

Thu, 16 Apr 2009 01:12:00 +0000

Day Three of the Craftsmanship Swap:

Micah and Paul had to head out of town for an Iteration Planning Meeting/Demo so it was just Colin Jones, Eric Meyer, and I in the office. We decided to do some tri-ping pong development on an internal product 8th Light is planning to release soon. So Eric would write a test/spec, I’d make it pass and write another spec, then Colin would make my spec pass and write one for Eric and the whole process would repeat. An enjoyable way to work made easier by their 24inch Cinema Displays and wireless keyboards.

I’ve noticed that everybody shakes hands at 8th Light. A lot. When someone arrives in the morning the new person tends to shake the hands of all who are already there. The same thing happens when someone leaves for the day. I asked Colin and Eric about the origins of this but they had no answers for me. It seems kind of weird, but it’s also sort of a cool affirmation of camaraderie. It may just be better than the grunt I give people when they leave for the day.

At some point I found myself arguing for these lines of Ruby today:

return false unless coupon = Coupon.find_by_token(session[:coupon_token])
return coupon.active?

Instead of a 5-6 lines of if/else code. Have I become the very person I despised when I got into Ruby? That guy who can’t rest until lines of code have been reduced to unreadable terseness? Even inject doesn’t look so bad to me anymore…

Of course, if we were using a current version of Rails we could do this:

!!Coupon.find_by_token(session[:coupon_token]).try(:active?)

Now that’s teh hotness.

Second Day of the Craftsman Swap

Wed, 15 Apr 2009 01:31:00 +0000

Day Two of the Craftsmanship Swap:

The day started off with a bit of work on slim for C with Doug Bradbury. Slim is a replacement for Fit which underlies Fitnesse and you need to write one for every language you plan to use FitNesse on. I gave it my best shot , but then Uncle Bob (Bob Martin - father of 8th Light founder Micah Martin) and James Grenning showed up. Great to see those guys – I don’t think I’ve seen James since my days of apprenticeship at Object Mentor. They are crazy better at C than I am so they worked with Doug while I paired with Micah on drving that ui through Fitnesse (which I talked about yesterday). We made some pretty good progress but he had some meetings to attend so I got to work with 8th Light’s newest apprentice: Colin Jones. Guess what he was working on?

Tic Tac Toe.

That takes me back. 5 years ago almost to the month, Micah had me write a Tic Tac Toe program to see if I had the skills to become an Object Mentor apprentice. Well Colin cleared the same bar and now he’s moved on to 3 dimensional Tic Tac Toe. Think of a Rubick’s cube with X’s and O’s inside the cubes. Three in a row still wins, but in any direction. Turns out there’s about a trillion different outcomes to a 3D cube so brute force solving is not going to work unless you have some time. Looking 4 moves ahead took about a third of a second, and 5 moves took 6 seconds. Looking at all possible moves and then choosing ones that lead to victory would take insane amounts of time. So we brainstormed some alternatives. You could let the computer chug away and build a massive database of all possible solutions – not super practical. Or you could approach it like chess: Have some pre-defined moves for the early game and a way of evaluating positions for the mid to late game. That way you can limit the depth of your search and still make ok decisions. We talked for awhile about what positions are valuable, but along the way we realized that 3D Tic Tac Toe is a terrible game. If you move first and take the center of the cube you’re going to win in about 5 moves. If you don’t take the center of the cube your opponent will and they’ll take slightly longer to win – but win they will. Draw it out for yourself and see. When Micah got back from his meeting we chatted about this and agreed that although there’s merit in learning how to tell a machine to play 3D Tic Tac Toe, some of the fun of doing it is spoiled by the game itself. So Colin’s switched over to 4x4 Tic Tac Toe (or is it Tic Tac Toe Tim?) for now but may return to the third dimension sometime in the future.

Btw, Micah’s cell phone ring tone is “Damn it feels good to be a gangster.” It does indeed.

First Day of the Craftsman Swap

Tue, 14 Apr 2009 02:20:00 +0000

Day One of the Craftsmanship Swap:

Today Micah and I paired on test driving a user interface through FitNesse. The UI in question is a JRuby rich client application that had reached the point where they wanted to have some acceptance tests that fire it up, click around, and assert that things are where they should be. We used DRb to stow away a reference to the application when it started up and then we could make the application dance. If you haven’t used DRb it’s yet another deceptively simple and powerful Ruby tool. Check this out:

~ $ irb

> require ‘drb’
=> true
> object = “hello”
=> “hello”
> DRb.start_service(‘druby://localhost:9001’,object)
=> #
> ref = DRbObject.new(nil, ‘druby://localhost:9001’)
=> #
> ref.size
=> 5
> ref.to_s
=> “hello”

Stick any object in and then another Ruby process can pick it up and do stuff with it. Neat.

I noticed 8th Light has the same desk space problem as Obtiva in that we have an open workspace, but people tend to settle into one space. And then they let all their personal junk pile up. Where do you keep all the personal products (toothpaste, dumbbells), mementos (awards, action figures, ninja swords (interesting side note: Every 8th Lighter got an engraved ninja sword for Christmas)), and other detritus (old specifications, note cards) when people don’t really have a desk of their own? When I went up to Atomic Object a few weeks ago (I gave a talk at their XP West user’s group) they had a bunch of cubby holes, like you might see in a school, big enough to toss a backpack into. At least it got the backpacks off the floor. Personal space is hard when you’re committed to pairing. I’ve seen situations where developers had their own desk in addition to working in a ‘war room,’ but that can be tricky as when you’re not used to the war room it can be tempting to sneak off to your desk so you can ‘get some work done.’ Phone calls can also be a problem. I love the pairing and the war room environment and there’s no way I would have learned as much as I have without them, but they do present some unique problems.

I finished up the day having a good time ping-pong pairing with Eric Smith on a large rails app. It’s funny how quickly you can pick up the spirit of an application and contribute when conventions are followed and there’s an expert engaging you at every turn. Sure, the first 30-45 minutes I was asking a lot of questions but after that I was adding something to the process – it really does surprise me every time.

Craftsman Swap Begins Today

Mon, 13 Apr 2009 12:16:00 +0000

This morning instead of taking a train to downtown Chicago I’ll be driving to the northern suburb of Libertyville and working at 8th Light for a week. Obtiva (my company) is participating in a craftsman swap so 8th Light will be sending someone down to Obtiva whilst I work with 8th Light. This should be pretty interesting as the last time I worked with some of the 8th Lighters I was an apprentice at Object Mentor.

Easy Charging of Credit Cards with Spreedly

Tue, 24 Mar 2009 00:36:00 +0000

When we built SQuiD (easy project management for link building), we had a problem in that the client was on a limited budget and yet the site had to charge people subscriptions for usage. Which, if we wrote that part ourselves, would take all sorts of time to research, implement, and (most importantly) not screw up. When you take people’s credit card numbers there is no such thing as a small security breach.

Luckily my teammate, Joseph Leddy, remembered hearing about Spreedly -- all they do is “Make Selling Subscriptions Easy.” We decided to use Spreedly and it saved us a ton of time (and consequently money). Here’s 9 reasons why I loved working with Spreedly:

1. Free to start and test with!
Spreedly doesn’t charge any money to sign up and run tests against their API. You start paying only as soon as you’re ready to start collecting real money.

2. Ridiculously test friendly
You can create a pretend (test) area that you can charge accounts to. It looks and behaves exactly like the real version, even down to the managing of your pretend subscriptions. When you’re running the site on your local machine you can follow a full path to Spreedly (with stubbed out credit card processing) and back to your local server.

3. Super easy Rails integration
This the all the code I needed to integrate with Spreedly:

class Subscriber < ActiveResource::Base
self.site = “https://secretapitoken:X@spreedly.com/api/v3/test”
end

That’s it. Now I can call:

Subscribe.find(id)

and get back a Spreedly object I can interrogate.

4. No storing credit card numbers on my site
Don’t have to worry about purging them from the logs, encrypting them, storing them in a secure place, or whatever. My site never sees credit card data so I don’t have to care about it at all.

5. I don’t have to track time left on a subscription
Spreedly will tell me when the customer’s run out of time.

6. Nice notification when subscriptions have changed
I expose a controller action to Spreedly and they send me a list of Subscriber ids that have changed, when they’ve changed. If they can’t reach me through some fluke of the internet they keep trying until they do.

7. Good documentation
Very easy to follow.

8. Great customer service
They usually got back to us in 15 minutes or so. Really.

9. Versioned API
We launched with version 3 of the API. They’ve since gone to version 4, but since there’s a “v3” in that “self.site” above I don’t have to worry about that. I can upgrade on my schedule.

I'll be Speaking at Rails Conf

Thu, 19 Mar 2009 01:05:00 +0000

Just a quick note to let you all know that Rails Conf accepted my talk. It’s called “Using metric_fu to Make Your Rails Code Better” and you can find more details here. If you’d like to join me in Vegas for 15% off you can use this nifty code: RC09FOS

Hope to see you there.

On Going Fast Because You're A Start-up

Wed, 18 Mar 2009 00:57:00 +0000

Today, for some reason, I was thinking about a start-up I worked on few years ago and how we could go really fast because (almost) no one used the darn thing.

When you’re working on a small budget you need to get that code out into the wild as soon as possible so you can figure out if this idea of yours has any legs. So maybe you test a bit less then you would with a big application and you don’t pair program that often and workflow is managed through IM and email and you take shortcuts because if you didn’t the code would never see the light of day. That’s the way it has to be and everyone accepts that.

However, there will come a reckoning – and it’s not fun from the owner’s point of view.

The site’s popular – it’s making some money but your development team is slowing down and asking for expensive things. They’ve brought on a new guy, but in addition to costing more money the team (which has now ballooned to two people!) is going slower for some reason. Things that used to take an hour take 1.5 – and through the miracle of pair billing they cost 3 hours!

The team used to deploy to production whenever they felt like it, but because of some recent costly screw ups the devs are throwing around ideas like: staging servers, regression suites, a DBA review of the SQL queries, and refactoring of complex code. Expensive ideas.

Work flow used go like this: Entrepreneur gets idea » He emails/IMs developer » developer codes and deploys. Now the team wants you start using some weird IM like thing (say Campfire) and a project management tool (say Lighthouse). And they cost money!

All this really sucks. The money that’s coming in just seems to get gobbled up by all these problems. And the thing the owner has a hard time seeing is that a long time ago everyone on the team made an assumption: This thing’s not a freaking bank so we don’t have to treat it like one.

A bank application has to work ALL the time and the cost of bugs can be EXPENSIVE. So there’s very good reason to spend some money being cautious. What has happened to the entrepreneur is that his tiny app has become his bank. He’s got years of time and tons of money invested in this website now. Failures 2 years ago didn’t have near the importance they do today. All the technical debt accrued along the way needs to be payed down. Now, most of these costs are one time hits with a much smaller continuing maintenance fee and they save way more money then not doing them – long term. But the transition from small to big has serious mental and physical hurdles to overcome. Mental in that everyone has to stop treating the app like it isn’t a big deal if it goes down. Physical because it’s going to need more people, servers, and products to keep it running. Those are some tough hurdles to jump.

Get Better Page Rankings with SQuiD

Thu, 05 Mar 2009 04:55:00 +0000

For the last few months I’ve been working on SQuiD: A website that helps you get a better Google ranking through totally honest means. There’s a lot of not so honest and even downright nefarious ways to try and increase page rank, but this site takes the high road and focuses on link building. Link building is way of taking a look at your competition and getting the places that promote them to promote you – by asking nicely. This may seem naive but it works, builds your reputation, and doesn’t leave inky black stains on your soul.

There’s a lot to cover about how the site works so I’ve created my first ever screen casts and you can find them here:
SQuiD Tour Part 1
SQuiD Tour Part 2

I should warn you that we charge for using SQuiD so if paying money for goods and services isn’t your thing then keep on walking, mister.

In the coming days and weeks I’ll be blogging more about the technologies and processes we used to develop SQuiD (Teaser - we used Spreedly to process credit cards and it really saved us a lot of time and money).

Metric Fu Now Includes Flay, Roodi, and Reek

Tue, 27 Jan 2009 02:39:00 +0000

I released metric_fu 0.9.0 last night and just look at all the super cool changes since 0.8.0:

Flay task finds duplication and structural similarities in your project
Reek identifies common code smells
Roodi finds mistakes and troublesome code
A configuration class so you can configure like the big boys
Standardization of the reports look and feel
Increased RSpec coverage
Source control type is auto-detected for Churn task
Under the hood refactoring to make adding new metrics reports easy

Of course it still does all the old stuff like 2 different types of complexity reports, code churn analysis, code coverage, and good old Rails stats. Set up a Cruise Control task to generate metrics reports every day and then you pinpoint the exact moment your project went to hell (or possibly avoid that fate – your call).

Check out the project home page at:
http://metric-fu.rubyforge.org/

The best part about this whole thing is that almost everything added since release 0.8 has been added by the metric_fu community – check out this impact graph on github.

Thanks go out to: Petrik de Heus, Andre Arko, Jay Zeschin, Lars Hoeg, Andy Gregorowicz, Sean Soper, and Bastien

Sprinkle Some Integration Tests into a Mock Heavy Test Suite

Wed, 03 Dec 2008 01:22:00 +0000

I like mocks and stubs and I use them liberally in my tests. This practice, however, is not without danger: If my models, views, and controllers are all tested in isolation then how will I know if a model change busts a controller or a view? Here’s something I like to drop in at the end of every controller spec:

1
2
3
4
5
6
7
8
9
10
11
12

describe JerkStoreController, “with views integrated and real objects” do
integrate_views

it “should render the page without blowing up” do
store = JerkStore.create!(:name => “Costanza’s house of bargins”)
store.products « Jerk.create!(:name => “Puddy”) « Jerk.create!(:name => “Newman”) « Jerk.create!(:name => “Steinbrenner”)

post :show, :id =\> store.id     end   end

Not a mock or a stub as far as the eye can see and I’m even integrating views. Normally RSpec doesn’t render the view in a controller spec, but if you include “integrate_views” it will. So now if I make some changes that bust MVC integration my spec will let me know. I don’t try to exercise all the possible paths with these tests – rather they are a safety net for problems isolated specs won’t detect.

Rails Rumble, Apprenticeship, and a Running Website

Thu, 27 Nov 2008 15:20:00 +0000

Apprenticeship is a big part of Obtiva (the consultancy where I work) so I introduced my friend Leah to Obtiva, they hit it off, and now she’s an apprentice in the studio. Now apprenticeship at Obtiva is a paid, billable position. She’s pairing everyday on a medium sized Rails business skill development project and learning a lot, but what just blew me away was the development of the “Run. Track. Run.” project.

Leah is a runner and for years she’s been talking about how she’d like to create a decent running website that could import Gamin GPS data and produce useful graphs for runners. Apparently all the sites out there plot every single point of data returned from the GPS device, with little-to-no smoothing, and so the graphs are extremely noisy and hard to read. Wouldn’t it be great if an actual marathon runner designed a website for runners? At which point I would usually say: “Yeah, that’s a great idea – we should do that someday.”

Here’s the cool thing: A few months ago the developers in the studio were talking about putting together a Rails Rumble team, Leah brought up her idea for the website, the Obtivians loved it, and now it exists. Go see it right now if you want: Run. Track. Run. That’s awesome. Leah had never written a line of Ruby code before this summer, but she got to be on a team of experienced developers (and one talented designer) who helped her implement a very cool website. She didn’t pay these people – they all volunteered to do it because they thought it would be fun to spend 48 hours building a website from scratch (which is the idea behind Rails Rumble). So in addition to getting paid to learn Rails as a day job, Leah got to build a website that had been rattling around her brain for some time now.

That sort of thing makes me proud to be an Obtivian and proud of Leah for implementing her idea.

If you use Mocha and RSpec then read this

Tue, 25 Nov 2008 13:29:00 +0000

I like RSpec, but I’m not a huge fan of it’s built in mocking framework. So, when I have the choice, I swap it out for Mocha. However, I really miss mock_model. If you haven’t used it, mock_model is an RSpec method where you pass in an ActiveRecord object and it stubs out a whole bunch AR magic so you don’t have to. This is crazy useful when testing controllers because when you controller has a line like this:
redirect_to(@model)

I don’t want to dig through a ton of Rails code to figure out what I need to stub on this model, I just want it to work. But I just found out that Mislav Marohnić has written a plugin that implements mock_model for Mocha – so now I can get the best of both worlds. You can find it here:
http://github.com/mislav/rspec-rails-mocha

Thanks Mislav.

Naked Hashes are Trouble

Mon, 24 Nov 2008 13:31:00 +0000

So I had this form in a Rails view that needed some changes so, amongst other things, I changed this:
form_tag(:action => ‘search’)

to this:
form_tag(:action => ‘show’, :method => :get)

I had noticed that the show and search methods essentially did the same thing (and had the same views) – which I why I refactored. However, when I made the change I got a 500. I was using restful routes and this form needed to be a get – which I thought I had indicated. But I had not – what should have put in the view was this:

form_tag({:action => ‘show’}, {:method => :get})

There’s this idiom in Ruby where we don’t put curly braces around hashes if we don’t need to, but I’ve consistently lost a bunch of time to mistakes like the one above. So I’m thinking about explicitly denoting hashes with curly braces in my future Ruby code just to improve intentionality.

Photos from Ruby Conf 2008

Thu, 13 Nov 2008 03:43:00 +0000

The Omni Resort was beautiful – this was our view during lunch:

I liked the Hotel but everything was very expensive. Breakfast buffet? 20 bucks. Awesome (really good bacon) but still 20 bucks. Dinner? 40-50 bucks. Cab ride there from the airport? 80 bucks.

There was a tanning bed convention being held right next door. They got a security guard in front of their door while we did not.

The next 2 pics are of the lazy river. I did a couple of laps (assisted by the current) and it was very nice.

A fountain and one of the many pools.

During the Matz Q&A my friend and fellow Obtivan Andy Maleh got up and asked some questions.

Ruby Conf 2008 Last Day

Wed, 12 Nov 2008 01:25:00 +0000

A wonderful, awful idea: ruby in the browser (and oh by the way it actually works!) by Christopher Nelson

Chris thinks that while Javascript is an excellent programming language, he likes choices so why not provide people with the option to run Ruby in the browser? Also, while programming rich web apps he found that oftentimes he ended up with business objects in Javascript and in duplicate objects in Ruby. Not very DRY. He wanted to be able to use his Ruby logic server-side and in the browser.

Two ideas that didn’t work out:
JRuby - the java security manager stopped him from doing just about anything
Silverlight wasn’t available for his operating system when he started the project

Rubyjs is what he ended up using. Rubyjs is a ruby compiler that uses ParseTree to produce javascript.
http://github.com/superchris/rubyjs

The translation is, of course, not without some changes:
Strings become Immutable strings
Procs become Functions
Hashes becomes custom hash object

Chris then showed us a hangman game written in Ruby, then compiled to javascript. We peeked under the covers to look at the outputted Javascript and it’s not very pretty. Neither is bytecode, I expect.

You can do Class level meta programming. You can even do method_missing (which resource intensive) in Rubyjs.

He wrote a rails plugin so you can do this in your views:
<%= rubyjs “my_class”, “my_method” %>
The above line, when executed, will grab my_method out of my_class, turn it into a javascript file, write it out, and load the javascript file in the view. It’s even smart enough to check last modified times to figure out if it needs to re-compile the method.

Microunit is his port of Miniunit so that he can test his rubyjs code.

Red is another Ruby to javascript compiler – he didn’t use it, but it’s worth looking into. He thinks its source code is easier to read, but it lacks some features of Rubyjs (no class level meta programming or method_missing). Red does have a very active community.

The Ruby Code Review. A Play in One Act by Jim Weirich and Chris Nelson

So this was a dramatization of a meeting between a consultant and client. Chris played an employee of MyTwitterFaceSpaceBook, who has brought in a consultant, Jim, to do a code review.

So they run the tests:
…EEE….FFF…EE…FFFFFFFFFEFEEEEEE….FFFFFFFFFFFFEEEEEEEEEEEEEE…FFFFFFEEEEFFFFFFFFFFFFFFFF……..EF
etc.

Chris - “We run the new_features_test rake task”
rake new_features_test:
………………

Funny.

Jim explained the broken windows theory. Chris then explained that they’ve been having problems with their fixtures. They use fixtures to test validations by having invalid fixtures. Which leads to problems with other tests when invalid data blows things up.

Jim put valid_options (which returns a valid option hash) right on the model – interesting. He says you can it somewhere else if you like. I would.

Jim recommends Continuous Integration like CC.rb. How do you cover the uncovered? Jim recommends doing it a little bit at a time. He comments out the whole untested method, then only uncomments enough to make the test pass.

While reviewing the code, they found some interesting metaprogramming – If a user has a friend named Bob Jones, then that user gets a methods called bobjones, which returns the Bob Jones object. Awesome. Unfortunately there’s a user named “Des Troy” that overwrites the Rails destroy method.

Chris and Jim had a great chemistry. Chris’ “But dude, it’s awesome ‘cause it’s metaprogramming” performance was funny and a little too real.

Seattle.rb Rocks! by Seattle Ruby Brigade

This presentation was sort of a lightning talk just for the Seattle Ruby brigade. Why do they get their own lightning talk session? ‘Cause they have 90 unique gems and over 400 gem releases between them. You can check out their many projects at http://rubyforge.org/projects/seattlerb/

Eric Hodel UPnP

Eric wanted to watch movies on his Playstation three through his laptop or maybe it was the opposite – he talked fast. So turning the PS3 into a media server? Or is the computer the media server? Anyway it looked cool.

JD Barnhart

Extended the Rad DSL to talk to the little circuit board that everyone loves: The Arduino. He made it do lots of stuff. Here’s a video of it linked to some servos hitting wine glasses with tiny hammers:
http://www.vimeo.com/1272402

Here’s a cool LED tower controlled with resistive strips
http://www.vimeo.com/1582919

(My buddy Josh Cronemeyer has also been messing around with the Arduino:
http://www.cuberick.com/2008/10/minchia-arduino-e-grande.html)

Phil Hagelberg Bus Scheme

What is Bus Scheme? It’s Scheme implemented in Ruby while riding on the bus to work. I usually just listen to podcasts.

Ryan Davis
Confessions of a Ruby Sadist - the short version:

I like to hurt code
People will press charges if you hurt them
But code won’t
Make your code your bitch

He’s written a few projects that he uses to keep his code in line:
Autotest runs your tests behind the scenes while you code - you get notifications when you break stuff right away.

Heckle mutates your code and runs the tests again. Mutated code better not pass the tests.

Flog measures the complexity of your code.

Flay finds structural similarities in code that are ripe for DRYing up.

Aaron Patterson
He wrote Nokogiri - Which is basically a lot like Hpricot but faster. And then _why the lucky stiff heard about this speed boost and re-wrote Hprocot to be faster than Nokogiri:
http://hackety.org/2008/11/03/hpricotStrikesBack.html
Oh, it is ON.

Ruby Conf 2008 Second Afternoon/Evening

Sat, 08 Nov 2008 13:34:00 +0000

What Every Rubyist Should Know About Threads by Jim Weirich

Concurrency is becoming bigger as computers get more cores. If you look at a graph of clock speed they flattened out in 2003.
Past performance gains:

clock speed (not so much anymore)
execution optimize
cache results

So if clock speed is not going up at a rapid pace anymore, now what?:

hyperthreading
multicore (the new saviour)
caching

Applications will need to be concurrent to exploit multicore machines. 100 CPU machines are on the horizon. Race conditions are trouble:

If thread one has this:
shared_variable += 1

And thread two wants to do the same thing:
shared_variable += 1

What if the first thread looks up the shared_variable and find that it’s 25 but before it can change it the second thread reads the value (still 25 because it hasn’t changed). Now the first thread will add one (shared_variable becomes 26) and write the result back to memory, then the second thread will add one to the stale number (25) and write 26 back to the shared memory. The output of those two threads running should have been 25 + 1 + 1 = 27 and yet the output was 26. Not good.

To fix this you need to disable context switching when touching a shared object and then re-enable it afterward:

m = Mutex.new
m.synchronize do
shared_variable += 1
end

Now all is well. However, this is trickier than it sounds – if you have multiple objects interacting it’s difficult to see where things must be synchronized. Also, you can get into situations where a circular relationship causes a deadlock:
1 waits on 2 waits on 3 waits on 1
Nothing can happen because threads are all waiting on each other.

To make your project thread-safe you must:

Protect every shared memory access with a synchronize
Be aware of extended situations
Have a strategy in place for deadlock
Evaluate every line in every library you use to make sure they follow the above steps

To sum up: Threading is hard.

Jim gently pointed out that if you really want to avoid race conditions you might want to use a language that was designed from the ground up to avoid these problems like Erlang, Clojure, or Haskell.

Using Metrics to Take a Hard Look at Your Code by Me

Good crowd. Everyone seemed pretty interested and there were a bunch of questions. I consider the talk a success. Except that I left my video adapter on the stage. And my laser pointer slide advancer thingy. What? I was nervous!

Ruby Kata and Sparring by Micah Martin

Micah used to be very serious bout martial arts and he thinks the becoming a good developer has some parallels with becoming a good martial arts practitioner. Micah’s martial arts teacher didn’t look very impressive, but he was a master. You’d be on the floor before you knew the fight had started. He got there through disciplined practice. To become a master coder you need to practice outside the office.

Kata - detailed choreographed patterns of movements. In matial arts you practice series of punches, kicks, blocks, etc.

Dave Thomas:
http://www.codekata.com

Uncle Bob:
http://butunclebob.com/ArticleS.UncleBob

What about performance? Every time you try for a belt you perform your Kata. This does three things:
The publicness of the performance motivates the student.
It’s also a measurement of skill.
And it shows respect for your art.

Micah then performed a live coding kata for us. He has been practicing implementing Langston’s Ant in Ruby. He bowed before starting and then cranked out some code TDD style.

Sparring is common in martial arts. In Ruby you have the Ruby quiz and the Rails Rumble. Micah announced another contest starting… Now!

The Battleship Tournament ends on Ends Nov 30, 2008. The challenge is to build a computer AI to play Battleship, but winning against other AI’s is not everything. The winner will be determined by winning matches and the quality of the code as measure by various metrics. If you’re interested check out these URL’s:
http://sparring.rubyforge.org/battleship
http://github.com/slagyr/battleship

After having a number of drinks at the Pivotal Labs party I finally made to the hotel’s lazy river (which is sort of a simulated river that goes in a big circle). Drunkenly doing laps around some palm trees while resting in an inner tube is a good time.

Ruby Conf 2008 Second Morning

Fri, 07 Nov 2008 16:27:00 +0000

Aristotle and the art of software development by Jonathan Dahl

How do you identify a good programmer? Jon says Ethics. Ethics is about how you live your entire life. He thinks the what makes a good software developer and what makes a good person have parallels in their answers.

Kant – Only act on principles that you would like to become universal law. Kant would have loved Haskel.

There are principles in software, but sometimes they conflict such as DRY vs write understandable code. DRYing something up can make it hard to read.

John Stuart Mill
Utilitarianism – what matters is the effect of the action. The ends justify the means. However, It’s hard to know whether the effects will be good ahead of time. The Pragmatic Programmer is a good outcome of Utilitarianism. But Utilitarianism may lead to sloppy code and processes. Or the Cowboy coder.

Aristotle - ethics as virtue. The person is the important part. For Aristotle:
ethics == a life well lived == happiness == virtue

each virtue is between two extremes:
rashness > courage > cowardliness

Aristotle says that you become good by doing good. Which is kinda of circular. To become a good programmer, hang around the good and look at their code.

Jon thinks that Aristotle would like Ruby because Ruby is written for us as programmers. Haskel is based on math and Java is for business requirements but Ruby is for people. Also, to harken back to Matz’s point from yesterday, Ruby is between two extremes:
Lisp > Ruby > BASIC

Fear of Programming by Nathaniel Talbott

A significant part of what we do as programmers is art. And we can learn from artists who aren’t afraid to talk about their feelings. At this point

Fear is an emotion that effects our productivity.
Fear of a blank page and a blinking cursor.
Fear of existing code:
Legacy code that’s hard to understand and work with.
Good code can inspire fear as you might mess it up.

Nathaniel then walked around the room, Oprah style, and had people volunteer things they are afraid of:

Fear of not finishing.
Fear of putting it into the wild.
Fear of loss of excitement about coding.
Fear that what you write will be useless.
Fear that someone has written it better before you and you just haven’t found it.
Fear that my imagination outreaches my ability.

Fear is good as a warning mechanism, but pathetic as decision making tool. You can manage fear by learning. Much fear comes down to a fear of the unknown. Testing is an antidote to fear and it’s a great way to break a problem down.

“The War of Art” and “Art and Fear” are two books Nathaniel recommends on this topic.

How do you, the audience, deal with fear:

Pair programming.
Talking it through to someone else.
Tackle the worst thing first.
Write fears down on paper.
Taking a walk.
Writing only the test cases.

He ended with the idea that passion and love are the ultimate antidotes to fear.

Ruby Conf 2008 First Evening

Fri, 07 Nov 2008 03:22:00 +0000

Better Hacking With Training Wheels by Joe Martinez

“We all have a stake in each other” – we use each other’s code in many, many libraries.

What automatically checks the quality of our code? Joe wants to encode rules in a standard way (called wheels) so that you could drop it into Training Wheels and it would enforce your rules on the code. People who write new libraries could include a wheels directory in their library which would offer suggestions.

Interesting idea – on my current project we’ve be having problems with people calling destroy_all on an ActiveRecord class for no good reason. We could define a rule that shoots out a warning every time you do that.

http://github.com/capitalist/training-wheels

NeverBlock, trivial non-blocking IO for Ruby by W. Idris Yasser

“Neverblock enables concurrent DB and network access without thew need to change the program flow” Which is awesome.

Idris said that the Evented model is good but you must adjust your code while Neverblock doesn’t need any adjustments. So you can take your Rails code, require some files, swap out the mysql driver for the Neverblock one in the database.yml and you’re done. If you use Thin that is. But still very cool

Neverblock uses Fibers (new in Ruby 1.9) which are normal methods that can be paused and resumed at will:

f = Fiber.new do
x = 5
Fiber.yield x
x = 6
Fiber.yield x
x = 7
end

puts f.resume #=> 5
puts f.resume #=> 6
puts f.resume #=> 7
puts f.resume #=> Error, dead fiber called

Fibers use less resources than Threads in Ruby.

Neverblock supports PostgreSqL and MySQL (with the MySQLPlus driver)

Neverblock also supports sockets.

Neverblock needs Rails 2.1, but not quite compatible with 2.2 yet.

Neverblock can work with Ruby 1.8 – they implemented Fiber-like things using threads.

Lightning Talks:
A nice Japanese man got up to promote RubyKaigi (Ruby Conf in Japan). He said they’d like to have more non-Japanese speakers. It is happening July 17-19th 2009.

Duby – Charles Nutter
Statically typed Ruby hybrid. For speed reasons? Yes, it is faster.

Rubists you might not know
A Japanese programmer got up and talked about two Rubyists we might not know. One was the release manager of Ruby 1.9 and the other was Itojun - the IPv6 samurai The presenter was quite upset when he talked about Itojun as he had passed away last year. By all accounts he was much loved.

Ryan Davis - Flay
Flay looks for structural similarity. It finds code that is duplicated. But since it uses ParseTree, to look at the code, it can find code that does the same thing but look different. Which is freakin’ awesome. Use it to DRY up your code.

Ryan also wrote Change so you can change the class of a Ruby object on the fly. Which is evil and he admits that. In fact he revels in it.

David Chelimsky – cucumber
Cucumber is a re-write of story runner. It now uses tree top which means stories can be in any language. Also the BDD with RSpec book is finally coming out. Beta in December, real book in April.

The Nature of Truth - Yossef Mendelssohn

truth in Ruby:
true
42
0
‘this sentence is false
[1,2,3]
[]
All of the above are true

Not true:
false
nil

sudo gem install truthy
so now you can say:
0.truethy?
and know the answer. Finally.

Conferences - are really fun
and make you stupid

Introducing Flunch the gem that tries to re-assign getters to setters and vice-versa. Cause… Um, it was fun. But it didn’t work. However if you try stupid stuff like this you will learn a lot. I think that was the point.

Test-stack
Another testing thing. Seems to be a mix and match for various levels of Rspec, shoulda, rr, and other things so you can customize your tests to taste.

At this point my computer’s battery finally gave out. It’s about 10:21 and I’m going relax in my room for a bit before I go to sleep. My presentation is at 2:10 tomorrow.

Today was a long fun day.

Ruby Conf 2008 First Afternoon

Thu, 06 Nov 2008 20:50:00 +0000

I was thinking today that most people design their talk to last 50-55 minutes but I’ve noticed that conferences have shorter and shorter times slots. 40 minutes + 5 for questions is becoming the standard. So the end slides tend to go by pretty fast.

Another thought: Print the ID badge for the conference on both sides. Exactly the same information on both sides. Fully 50% of the people at Ruby Conf have their badges turned around to the white side and when I can’t remember their names (even though we were just introduced) I can’t cheat because all I see is a stupid white rectangle. Somehow GLSEC managed to figure this out, why can’t other conferences?

JRuby: What, Why, How…Try It Now by Tom Enebo and Charlie Nutter

Why JRuby? Well the JVM is awesome. 15 quadriliion man years of work or something. 15 quadriliion Sun man years can’t be wrong, right?

They have declared that Ruby 1.8 is “done” (again) and 1.9 mode can be triggered with a flag. Full 1.9 compatibility is coming.

Was it two years ago when I first saw them talk? Yeah, I think it was at the Rails Conf 2007. So about 1.5 years ago. JRuby was pretty impressive then and it has come a long way since.

Now that Rails 2.2 is thread safe you can really take advantage of that with JRuby. They showed a ridiculous graph comparing Rails 2.2 under load with MRI and JRuby. JRuby crushed the competition like a grape.

Btw: It’s kind of amazing how many people here have the brand new Macbooks and Macbook Pros. Those things just came out, like what, a month ago?

Ruby In the Clouds by Jim Menard

Jim uses 10gen which is a different stack indeed:

Babble is the App server? Mongo is the DB? Javascript is server-side?

Whoa, I walked into this talk thinking it would be another putting your ruby/rails app on the EC2 cloud. This is a totally different direction.

Other Features on 10gen:
Multiple server-side programming languages including
* Python
* Ruby
* JavaScript

Multiple server environments including
* Standard 10gen
* CGI
* WSGI

Multiple frameworks including
* Rails
* Django

Multiple deployment options including
* 10gen-provided hosting - deploy on 10gen managed infrastructure
* Run-your-own-cloud on your own infrastructure or hosting

The thing about this is that without ActiveRecord working, you’d have to code for 10gen from the start. Or face a very arduous conversion from ActiveRecord to their custom way of accessing the DB. Jim sez ActiveRecord support is coming soon.

The Big idea behind 10gen is that you could write your app and never have to worry about scaling it. Which is pretty cool.

MacRuby: Ruby for your Mac by Laurent Sansonetti

Everyone wants to develop Cocoa in Ruby. It can create awesome Mac apps. But you need to develop in Objective-C. Which is not so pretty. However, they both share some of the same concepts and ideas.

RubyCocoa exists, but it’s a bridge. So conversion is happening and that is slow and resource intensive. And each runtime has a different garbage collector!

MacRuby interprets Ruby on the Objective-C runtime. Kinda like JRuby does with the JVM.

All objects in MacRuby inherit from NSObject.

HotCocoa is a thin Ruby layer on top of Cocoa. He showed some code and it really looked like Ruby. Slick desktops apps for the Mac written in Ruby are coming for Christmas (the projected “Production Ready” date of MacRuby).

From the audience: “What about iPhone apps?”
Laurent: (said in an outrageous French accent) “I’m sorry I can not answer this question”

Time for a break – surely they’ll have low-carb treats, right? Maybe just this once?

Ruby Conf 2008 First Morning

Thu, 06 Nov 2008 18:47:00 +0000

Woke up 15 seconds before my alarm went off again today. Perhaps I’m a little keyed up?

Keynote - Matz “Reasons behind Ruby”

Matz’s presentation started out with the question: Why Ruby? He asked why even though Ruby has lots of problems, like all languages, why do we love it? Which is interesting. I’ve seen Matz talk a few time now and he always talks about the love of languages. The idea of loving what you do is never far from his mind. When Matz was young, his first computer was a pocket computer in 1980 – only 1400 steps allowed. Only 150 memory places. And the Language was BASIC.

Later he found Lisp and it was the opposite of everything in Basic – total freedom to do just about anything. And he loved it. That was, however, before he started using Lisp. When he started using Lisp in practice, he wasn’t having a good time.

Basically Matz felt that Basic took all the power from him, but Lisp gave too much back. He wanted to have a framework to guide him, but also to have power when he needs it. Which lead to Ruby.

As always, Matz’s talks are hard to sum up. You really have to experience one to know what happened.

Scaling Ruby (without the Rails) by Gregg Pollack

This talk was another one of Gregg’s great survey talks where he picks a topic and then reviews all the cool things going on in that area. And his Keynote-Fu is strong. This time he was covering tips and trick to make your Ruby code faster.

In a situation where you have a bunch of threads, each will get a slice of time. But anytime Ruby drops into some C part then it blocks all the other threads because it doesn’t know where it can pause the C stuff. Any IO operation is typically blocking. Making database calls is another common example of this. You can use Neverblock to have concurrent db calls.

If you have 4 CPUs and 4 threads you still only get 1 CPU. This is Green Threading and it really doesn’t use all your processors to their fullest capacity. If you want use all 4 CPU, then you need 4 processes. Many Mongrels is an example of this. Most Rails apps typically run a whole pack of Mongrels to take advantage of multicore processors. Unfortunately, now you have a full stack duplicated a whole bunch of times and the memory waste is huge.

Native threads are cool and coming in 1.9, but are not concurrent due to Global Interpreter Lock. Of course you always go to JRuby (which runs on the JVM and has true native threads).

Event Machine - a fast single threaded application. It runs one thread at a time until it’s done - this can be much faster (instead of each getting a slice of time). However slow requests can hold up fast requests. So Event Machine is best for fast requests. In Merb you can optimize certain requests to be deferred requests (if you know what’s slow). Then you get the best of both worlds.

DBSlayer sits between the DB and your processes. You can add more DB’s on the fly – which is kinda wild.

Message Queuing with Starling. If the server dies it can recover – nice. Very scalable as you can just add more workers.

Dropping into C – Ruby Inline lets you drop into C inline in your Ruby code if you need close to the metal speed.

You can use Ruby-prof to profile your code like so:
RubyProf.profile do
# some code
end

You can print out fancy html output and even show call trees.

Speed up Tricks:
Method calls are slow:
“Hello “ + @name + “, how are you?” vs
“Hello #{@name}, how are you?”
less calls == better
The first example is slower because of the two extra “+” calls

Destructive methods are faster than their non-destructive counterparts as they don’t create a new object. So gsub is slower than gsub!

If you are doing case switching then you should put the most popular one first so the other, less popular, conditions are not evaluated.

Rails 2.2 has built in support for memoize so you can save the output of a method call and return that response the next time the method is called.

Gregg asked all bloggers to mention that Envycasts are available online (including this talk). So I hope you’re happy now, Gregg.

Also you can go to RailsEnvy.com/rubyconf for the pdf of this presentation.

Simplifying Desktop Development with Glimmer by Andy Maleh

Andy wanted to create a multi-platform Ruby rich client application framework. His goals were to be concise and DRY, to only ask for the minimum info needed, and to value convention over configuration. He came up with Glimmer.

Glimmer uses Data Binding
The view and the model observe each other so that changes in one are automatically propagated to the other. Which is nice and effortless. An example he showed is where in the view he bound two different text fields to the same attribute on a model. Then when he typed in one field, the changes just showed up on the other. Without adding any additional code. This Blew My Mind as I am a guy who lives and breaths the web.

He showed the code behind the a simple TicTacToe game. It was concise and easy to read.

Home:
http://rubyforge.org/projects/glimmer/
Introduction:
http://eclipse.dzone.com/articles/an-introduction-glimmer

Glimmer will soon become an Eclipse project.

OK, off to lunch. Back at ya later in the day.

Great Lakes Software Excellence Conference

Thu, 06 Nov 2008 05:52:00 +0000

What a nice conference! Emphasis on nice. First Joel Adams, chair of Computer science at Calvin College, got up and welcomed us. Calvin College is where GLSEC was held and it looks so shiny and new I wondered if it had just been constructed. I stayed in a room the was about 200 feet from where I gave my talk.

Then the mayor of Grand Rapids, George Heartwell, gave us software dudes a nice little welcome speech. He implored us to see downtown Grand Rapids and then talked a lot about jobs and growth. I know Michigan has had hard times, but from the looks of the airport and Calvin College (which is about as scientific a survey as it gets) Grand Rapids must be a city on the grooooow.

The opening Keynote was given by Michael Cloran who is the the CEO of Interactions.

He started off with a live demo of the “Service Factory.” He spoke, through his phone, to a computer and it parsed his voice and answered his questions. It was scary impressive. Typical line from the extended dialoge he had with the machine: “I have a flight at 5 or 6 today from altanta to London, oh wait I’m mean chicago to London I was in atlanta yesterday and I was wondering if you could tell me the gate and exact times” And then it nailed the response. Whoa.

Of course then he revealed that the secret behind the fabulous voice recognition was real human beings. He went on to explain that this is part of what he calls “The second service revolution.” The First industrial revolution was about craftsman. The 2nd was assembly lines where each person did one thing. And thats the deal with his call center. Other call centers have employees who either know or look up all the answers. This takes lots of training – which is bad new in an industry with high turnover. Cloran’s company breaks up the calls and sends each person’s response to a different “Intent Analyst.” So an IA sits in front of a computer and gets a message on his or her headset along with some text on the screen describing the context. The IA then indicates to the computer what the person wants and the software does all the magic of providing the response while the IA is off listening to another snippet.

What’s really interesting is that about 60% of calls are assigned to 2 different IA’s, without their knowledge, and they get points if they both provide the same response. The one who gets there first is given double points. Their point average is directly linked to their hourly pay. Of course incentive pay inspires people to try and game the system so they’ve been constantly tweaking the system to eliminate “cheating.” An example Michael provided was that there are common mistakes new employee’s make so seasoned pros would quickly imitate those mistakes to get more points. They had to write algorithms to catch the cheating and penalize it with triple negative points. Reminds me of Joel Spolsky’s classic “The Econ 101 Management Method”

Of course, as a start-up they made their fair share of mistakes. In the beginning they made a proof of concept demo and then when they got funding they just kept modifying it instead of re-writing some very sloppy code. This cost way more time than it saved. And they didn’t write unit tests until the complexity threatened to overwhelm them. One of the biggest things he regrets is separating the QA and Dev teams. They thought that Sarbanes-Oxley demanded such a separation (later they found out it didn’t) and this separation lead to lots of inefficiency, anger between the teams, and morale problems.

They also had IT problems. In traditional IT you can delay something as long as you want without much penalty. However, if you approve something that fails, you’re in huge trouble. What incentive is there to do something fast? Need to analyze risk and reward, put it in front of the business, and then not freak out when trouble comes.

All in all, one of the most interesting keynotes I’ve heard in a long time.

“Encapsulated Process Objects” - Jeff Dalton

Jeff’s deal is that he combines Agile and CMMI into one thing. Which is interesting because CMMI is usually associated with non-agile waterfall projects. Dalton stated that most devs see process as a ridged, audit-driven thing that demands slave-like adherence. But he doesn’t. He sees CMMI as much more flexible than most people realize. His “Encapsulated Process Objects” are meant to be a smorgasbord of choices you can pick from when developing a project.

Unfortunately he went kinda fast and used a lot of terms I wasn’t familar with. For example, when talking about “verification” in CMMI he mentioned that you could use either pair programming or Fagen inspections or test based design. Is Test based design the same as Test Driven Development? And what’s a Fagen? (insert Dickens joke here) When fulfilling Feature Validation you could use “Use Cases,” “Simulations,” or “Prototypes.” Uh, OK. Are use cases user stories? ‘cause I’ve used those before.

So I felt kinda lost. But I did make a mental note to contact him if I’m ever in a place that demands CMMI so I can figure out how to wedge in Agile/XP

Here’s a strange thing about GLSEC: I only saw about 10 other open laptops during the whole day. How do these people attend a session without a Macbook Pro to type on? The wifi was wide open!

“Taking the ‘Pro’ Out of ‘Process’: Test Process Improvement for Everyone” - Jess Lancasteru

This talk was about improving your QA team’s processes. Jess started out by saying that test process improvement is hard because you don’t know what to do, when to do it, or how to do that which you eventually decide to do.

Jess is a big fan of the Test Process Improvement Model (TPI) and recommends this book:
“Test Process Improvement: A step-by-step guide to structured testing”
as a way to assess and improve your testing processes.

I have to admit that during this talk I sat near a window, so as to be near a power outlet for my Mac Pro, and it was a beautiful fall Michigan day. 70 degrees, beautiful colors, trees waving gently in the wind. It was hard not to stare out the window. I may have missed some things.

Jess fessed up early to re-using an internal slide deck for this presentation and it showed. Some key terms weren’t defined and the points on the slide sometimes seemed a bit off of what he was saying. If you are interested in using TPI, then he recommends an excel spreadsheet found here:
http://www.sogeti.nl/Home/Expertise/Testen/tpi_downloads_uk.jsp
It helps you use TPI to score your testing process.

“The Art of Refactoring” - Kealy A. Opelt

Kealy started out her talk with a shout out to Martin Fowler and his seminal book “Refactoring”. She defined refactoring thusly:
Refactoring improves design but does not add observable behavior.
And she cautioned that Refactoring is dangerous without unit tests.

Why refactor?

Improve design
Make code easier to understand
It helps you find bugs
It helps you program faster

The rest of the presentation was workshop style in that she would identify a code smell (say “Long Method”) and then use one of the prescribed cures to make it better (like “Extract Method”). She even gave us real paper handouts on which we could practice with our neighbors – I haven’t done worksheets since my teaching days. Wait, do W2 forms count as worksheets? Or 1040 forms? Anyhoo, it was some old school fun.

Other examples were:
Feature envy – when an object is interested in another object much more than itself. Solution: Move methods into the envied class.
Data clumps – a bunch of variables that have to do with each other. Solution: Create a class to hold that data.

“Using Metrics to take a Hard Look at Your Code” - Me

The talk went OK. It’s a presentation that shows a fair amount of Ruby code and the crowd had only a few Ruby users so that may have put a damper on things. Also they didn’t really laugh at any of my jokes. But I soldiered on and I think a few people got some value out of it. A software development track at an XP/Agile conference is an interesting thing in that most people are going to be out of their element. Much as I was confused by the CMMI guy’s jargon, my audience had to struggle with some unfamiliar terms in my talk. In a way, it’s a good thing to be exposed to new ideas. But in another, much more real way, it can be a little depressing.

“Moving from 1.0 to 2.0: How We Brought an Application From Client-Specific to Generic and Marketable” - Sam Williamson

This talk was about how a startup made all the wrong decisions early on and yet managed to right the ship and get some working software into the wild. And then modify it to be generic enough to be sold to many different companies.

Sam is part of a small distributed team and he recommends: http://www.jingproject.com/ to make quick screencasts that can be viewed on the web. They used Jing daily during standup to show what they had been working on the day before. Sounds like his standup was a little more heavyweight than the ones I’m used to but that makes sense when everyone works in different cities.

Early on they had no source control, no tests, and no consistent way to build the project. After getting kicked around a bit, they added in tests, visual source safe, CruiseControl.net, and agile practices (they had to do remote pairing with screen sharing).

They had to interface with lots of different systems. From a shared drive on a mainframe, to a database, to an actual service. They wrote interfaces to wrap all of the services to minimize the coupling. Not only did it insulate them from changes in the third party systems, they could now go to other clients with different systems and all they had to write was a new bit of code that used the interface. Ha-Zaa for good design!

Last night when I got off my plane from Chicago, I realized that I would be in Grand Rapids for less than 24 hour so I could check in for my flight to Detroit (and then Orlando) before leaving the airport. What a jet-setter I am. I have to say that I always anticipated that being a jet-setter would be more glamourous.

“Agile is More than Just Makeup: Going the Distance with XP” - Michael Swieton

Practices matter. There’s a tendency in crunch time to say “screw pairing, we need to go fast.” but abandoning your practices during troubling times “is like turning off the light just when it gets dark” – Ron Jeffries

This was pretty general talk about agile practices, testing, mocking, and dependency injection.

I’m kinda running out of steam here. It’s 10:28pm and I’m in coach on my 3rd flight in the last 30 hours so while Swieton’s talk was good, I’m giving it a bit of the bum’s rush.

Jason Huggins - Final Keynote

There’s a another keynote to discuss? Ah hell.

But seriously folks, Jason is the creator of Selenium (a web testing framework that is definitely not named after the cure for Mercury poisoning), a former colleague of mine at ThoughtWorks, a recent quitter of the Googleplex, and now trying to get together a start-up. His new venture is called “Sauce Labs” and is dedicated to putting web testing on the cloud. Instead of running 1000 click through your website tests in serial, he’ll help you start up 1000 servers on EC2 to run crazy fast.

Jason’s talk was mostly about how the transition from silent documentation (another name for a manual) to screen casts is similar to the transition between silent movies and the talkies in 1927. He talked about how David Hanson’s Rails screencast in 2004 changed the game for documentation in software and we’re only just beginning to realize the fallout. Huggins really digs Castanaut for automated screencast generation even if it does talk in a scary computer voice.

All in all, GLSEC was a good time and I’m glad I was able to attend. The hotel and conference staff were really nice and very helpful and so I’d like to thank them here. In a blog they will never read. OK, time to get some airplane sleep for tomorrow is Ruby Conf!

Here I am in Grand Rapids, MI

Wed, 05 Nov 2008 02:34:00 +0000

Flew in on election night for the Great Lakes Software Excellence Conference. I’ll be giving my metrics talk tomorrow and then I’m off to Orlando, FL to give that talk at Ruby Conf. GLSEC is more of an XP/Agile/Process conference and I’m excited to be attending. The last time I was at a similar conference was XP/AU (Extreme Programming/Agile Universe) in 2004 when I was an apprentice at Object Mentor. I was just about to go back to my job as a high school physics teacher, but I did have an application in at ThoughtWorks. Ah, memories.

Metric Fu's Churn Now Supports Git

Tue, 07 Oct 2008 04:18:00 +0000

I just released version 0.8.0 of MetricFu. New in this release:

Source Control Churn now supports Git (thanks to Erik St Martin)
Flog Results are sorted by highest Flog Score
Fix for a bunch of ‘already initialized constant’ warnings that metric_fu caused
The flog reporter can now handle methods with digits in their name (thanks to Andy Gregorowicz)
Internal Rake task now allows metric_fu to flog/churn itself

To use the Churn task to find files that change a lot with Git, put this in your Rakefile:

MetricFu::CHURN_OPTIONS = {:scm => :git}

MetricFu is a Ruby Gem that gives you a fist full of code metrics. Find out more at the home page.

Subversion Binary Image Conflict Solution

Thu, 02 Oct 2008 01:04:00 +0000

Today it was my turn to merge in some bug fixes and updates into the current branch. And we had a conflict on an image file. What to do? Well, if you want to keep the incoming change then use ‘svn resolved’ or if you like the old one then ‘svn revert’ Yep, I totally knew that and didn’t spend an hour looking around the internet only to find a solution that only applied to svn 1.5 which doesn’t work with svn 1.4

Merging sure is fun.

Nail the Small Stuff

Tue, 30 Sep 2008 01:12:00 +0000

I was talking to a friend of mine the other day and he mentioned that his wife had been turned down for some dentistry that she needed performed. It turned out he had been paying for family dental coverage but his company had only signed him up for single. Once the mistake was realized, his company corrected the error. He might have to re-fill out some forms, and her surgery will be delayed, but all will be put right.

I mean it was a simple mistake, how could he possibly hold it against them?

You can spend lots of dollars on big gestures, perks, and what-not but it can all be wiped away by a bit of faltering on the basics. People need to know that the ground beneath their feet is solid – That they can trust the people who employee them to take care of certain details. When that trust is shaken they start asking themselves questions about other things.

It reminds me of Maslow’s hierarchy of needs:

Maslow’s hierarchy of needs is often depicted as a pyramid consisting of five levels: the four lower levels are grouped together as being associated with Physiological needs, while the top level is termed growth needs associated with psychological needs. Deficiency needs must be met first. Once these are met, seeking to satisfy growth needs drives personal growth. The higher needs in this hierarchy only come into focus when the lower needs in the pyramid are satisfied. Once an individual has moved upwards to the next level, needs in the lower level will no longer be prioritized. If a lower set of needs is no longer being met, the individual will temporarily re-prioritize those needs by focusing attention on the unfulfilled needs, but will not permanently regress to the lower level. For instance, a businessman at the esteem level who is diagnosed with cancer will spend a great deal of time concentrating on his health (physiological needs), but will continue to value his work performance (esteem needs) and will likely return to work during periods of remission.

Which is a long winded way of saying that if you don’t nail the small stuff your grand plans are all for naught.

Sunday Nights

Mon, 29 Sep 2008 12:02:00 +0000

I used to be a High School physics teacher and so people often ask me why I left teaching and to become a programmer. You’ll get a different answer from me every day you ask that question, but today’s answer is Sunday nights.

When you find yourself picking fights with your wife, kicking the cat, and generally needing a few drinks every Sunday night it’s because you know there’s a week of pain coming. This pain may or may not materialize but there’s a fair percentage that it will. Once, when I was a teacher, I had a 6’3’’ football player in my class who would sometimes take to getting up and screaming at me. One time, after an altercation about talking in class had turned into a huge blow up, he pushed past me on his way out of the room. Now getting knocked aside by someone who could probably kill me in a fight wasn’t a daily occurrence in teaching, but that’s not the point. The point is that after that day it was always a possibility in my mind. There were other altercations in my 7 years of teaching and each added a little more weight to my Sunday nights. Toward the end of my time as a teacher the mass was unbearable. My job was making my non-job time painful so I had to leave.

Angry Sunday nights mean your job is beginning to smell.

Different Ways of Installing Metric Fu

Thu, 25 Sep 2008 01:02:00 +0000

Something I didn’t mention when I announced the re-launch of metric_fu as a gem is that now, because it’s a gem, you have more options when installing it in your project.

First option:
Vendor Everything. Lots of people are big fans of unpacking gems into the vendor directory of their Rails project (or a similar folder in a non-Rails project) so that everyone who checks out the code gets the right version of the gem. You can then require it in your Rakefile like this:

require ‘vendor/gems/jscruggs-metric_fu-0.7.6/lib/metric_fu’

However…

Second option:
Conditionally require the gem. When I announced metric_fu back in April, a lot of commentors bemoaned the fact that they would have to install something into their production code that wasn’t really necessary for production. I don’t really mind doing this as metric_fu does not modify any classes: It’s mostly just a bunch of Rake tasks (and some report building code). However, I see their point. So if you don’t want to put metric_fu into your project, you can simply make sure the gem is installed on your CruiseControl machine and then conditionally require metric_fu in a rake file like so:

begin; require ‘metric_fu’; rescue LoadError; end

So anyone who has the metric_fu gem installed on their box can run the reports, while keeping everything else pure.

Stop your Mac from going to Sleep with Caffeine

Wed, 24 Sep 2008 01:45:00 +0000

I recently discovered Caffeine for the Mac and it’s pretty cool. When I’m giving presentations, listening to streaming audio, or performing some operation where I don’t want my mac laptop going to sleep I could mess with the system preferences… Or I could click on a friendly little coffee cup to tell my computer to stay awake. When the coffee cup is full, my computer won’t go to sleep or activate the screen saver. Empty coffee cup means my normal preferences are in control. Nice!

Many thanks to Todd Webb, for pointing this out to me.

Windy City Rails Part Two

Tue, 23 Sep 2008 01:17:00 +0000

Yesterday, I talked about the first half of the first ever Windy City Rails conference that happened on Saturday at IIT. Tonight, I’ll pick up where I left off with my summary of the presentations I attended.

Virtualization and Elastic Servers – Yan Pritzker of CohesiveFT

The gist of this talk is that the Rails stack is not well known so why not let CohesiveFT handle deployment for you. It’s a virtualization factory, that lets you put together a custom Rails stack and they can deploy and maintain it for you for as little as 10-15 bucks a month. Also, he pointed out that virtualization is a solution for your front end guys getting a working version. Which, having tried to get many a design person’s computer up and running, I think is a pretty cool idea.

Ten Things I Hate About Web Apps by Micah Martin

Short list of things Micah hates about Web Apps:

You need to know css, html, js, and the actual programing language just to get started.
Html – only 3 fonts to chose from
Css is not implemented the same on browsers
Javascript and will it be supported
Multiple browser support
The Photoshop dependancy (why should rounded corners and gradients be so hard?)
The statelessness of http
Flash/silverlight are not open source
Ruby on Rails – a pretty mask on top of the ugliness

His solution is Limelight -- A platform for writing rich ruby apps. It’s all Ruby all the way down. He showed off a simple demo of how easy it is to create apps with Limelight. Then he showed a tower defense video game his brother wrote using Limelight. It looked pretty cool, but it’s very audacious to propose a new platform.

Slight of Hand for the Ruby Man by Aaron Bedra, Relevance

I missed a bunch of this presentation ‘cause I’m a dumbass. The cool thing I did manage to see is “the-inspector” which is a gem that can tell you where some monkey patcher has redefined a method. So you can call this:

Inspector.where_is_this_defined {Klass.method_to_find(:something)}

Very cool. Check it out at: http://github.com/spicycode/the-inspector

The Morph App Space lightning talk was next.
Morph can help you deploy RoR in five minutes, it’s fully managed, and elastically scalable for $1 a day.

Rails Security by Aaron Bedra, Relevance

Aaron talked about using Tarantula to crawl your app and attack it. It’s particularly good at finding cross site scripting and sql injection attack vulnerabilities.

There are a number of solutions to preventing cross site attacks (aside from remembering to use ‘h’ everywhere):

His general recommendations for security:

Keep your framework up to date
Build security from the ground up
Audit your code

Test Last Development – Noel Rappin, Pathfinder

Regular TDD is Red-Green-Refactor and yet it often goes wrong and Noel spent some time talking about typical ways in which it does.

People often skip the refactor step – he calls this test driven mess as TDD is supposed to be about design
Data driven testing – fail the test, get the output, then put into test
Code then test – Which useful in exploration, but it’s easy to slip. It also leads to coverage gaps
Test last development – when you check in without tests and then later try to wrap tests around your code.

Noel uses two coverage suites:

One models only
And the other is everything else

He does this because controllers hit models a lot so they can hide lack of model coverage in tests.

Generally, I had a great time at Windy City Rails. I think Ray Hightower and everyone involved did an amazing job pulling this together.

Windy City Rails Part One

Mon, 22 Sep 2008 01:52:00 +0000

This weekend was the first ever Windy City Rails Conference and I was lucky enough to not only attend but also to speak. I thought I’d sum up the talks I attended in case anyone was interested.

After Ray Hightower, lead organizer of the conference, welcomed everyone Ryan Platte gave the first talk of the day: “Outside the sweet spot”

Ryan’s talk was about getting Ruby and or Rails into a large organization which is committed to things like: Windows, Oracle, etc. I.E. not the typical Rails stack.

In Ryan’s experience, Rails makes it’s way into such an organization the following way: Some guy picks up a Rails book and says “Hey, this is way better than the excel spreadsheet we’ve been using.” Then he talks one boss into the project, and maybe they hire some consultants to help. However the rest of the organization has issues with Rails (it’s slow, not compatible with their stack).

Ryan’s advice is:

Do top quality work quickly – get out in front of criticism by having good early results
Communication – talk to everyone
Model, model, model – Spend some time on the soul of your application
Get to know the legacy tech and why it’s there. Find out what will the people who wrote it feel about the changes? What will they hate. And what will they like. Get those arguments ready.
Use rest - it looks cool to the technical users

Warnings:

There’s a temptation to add all the cool whiz-bang stuff Rails can do/is known for. Throwing in the kitchen sink is a good way to doom your project.
DBA’s talk in SQL and will often times will refuse to run Ruby migrations.
While JRuby is cool - testing Ruby with object dependent on Java Objects means you’ll have to test in JRuby and spinning up the jvm is slow.

Up next was Rails Q&A with David Heinemeier Hansson

This first question was about performance in Rails. David said this has been a big effort for 37 signals over the last few months and his advice is to optimize http first: gzip, minify js (use yslow to find problems. There’s a google presentation about it you should see). There’s a common problem with devs, they think about code speed first. 80% of rendering time of a page is http stuff. Code is only 20%. They are working on putting these into Rails. Ex: timestamping of assets (gotta set up your server to use this).

David said that a 200 millisecond response is the sweet spot for the user to feel like your web app is fast.

He talked a bit about Rails 2.2 thread safety – good if you use JRuby or 1.9 (David thinks thread safety is optimizing the 20%)
I asked David whether Rails will be implementing any features from Ruby 1.9 that will force people to upgrade to a new Ruby. He said that there are absolutely no plans to abandon 1.8 support. There has been an effort to get Rails to run on 1.9 which is near completion. As for the speed boost of 1.9 (and YARV) he’s only seen a 20% speed improvement in real world Rails examples.

David wants to pull out validations into their own thing (and you can add them into your own models) and he spent a little time talking about that.

There was a question about observers and David said the he thinks they are kinda are old to him as he uses now uses memcached so he’s ripping out all his observers from his code. Expiring caches is a very hard problem and he really digs how memcached makes things easier.

Dave Hoover asked what David would have done differently if he had the last 4 years to do over again. David said he’s not real big on regrets. He went on to say that if you had tried to do all the things you now know you would have to do, you never would have started. It would be too daunting. When he started Rails it was a 1000 or lines of code. Very doable. David is not a big fan of retrospectives.

Internationalization was like the 3rd feature requested ever. And it’s a very hard problem. So they thought about it for a few years and let the people who really care about it sort it out. The Rails core team didn’t much care about internationalization and they weren’t experts. In this case the committee actually worked.

Asked about what he was excited about, David talked about memchached, http (the creators showed lots of foresight), and document databases (couch db and it’s ilk)

David said that people often make the mistake of thinking that 2000 users is hard and so is 1 million so why not go for 1 million. They don’t get that the scale of hardness is way different for 1 million. Basecamp was out for a year before it could even pay people’s salaries. You win by staying in the game. Many companies self destruct trying to get too big too fast.

Then it was time for Dave Hoover’s (of Obtiva) lightning talk. He admitted he was stealing from Uncle Bob’s 2008 Agile keynote as he talked about software craftsmanship. Which was weird as I was sitting next to Micah Martin – Uncle Bob’s son. Dave cited the analogy of doctors washing their hands vs getting started on surgery. There was a time when hand washing was viewed as a needless use of time for a busy doctor. And we face the same problems as developers: Deliver features vs refactor/write clean code. Obtiva is trying to be about craftsmanship.

Well, that’s all I feel like typing tonight. I’ll wrap up the rest of the presentations tomorrow.

Metric Fu is Now a Gem

Tue, 16 Sep 2008 03:40:00 +0000

I just released version 0.7.6 of metric_fu and there’s all sorts of new stuff:

First, MetricFu is now a Ruby gem on GitHub at: http://github.com/jscruggs/metric_fu

Also, the Flog task can now flog any set of directories you like. Just put this into your Rakefile:

MetricFu::DIRECTORIES_TO_FLOG = [‘cms/app’, ‘cms/lib’]

And, the Flog reports are much nicer:

The Flog report generator now looks at a MD5 hash of the files its examining to determine if they files need to be re-Flogged – which saves time.

The Saikuro report generator can also look at custom directories like so:

MetricFu::SAIKURO_OPTIONS = {“–input_directory” => ‘“cms/app

cms/lib”’}

MetricFu started its life as a Rails Plugin, but I’m trying to turn it into a flexible gem that can be used in any Ruby application (but still have defaults that enable easy use in a Rails project). Eventually I’d like to create something that could tell you about methods that have high complexity (Flog and Saikuro scores) but low test coverage. But that’s going to be a challenge. Feel free to help, if you like.

I’d like to thank Sean Soper for contributing a ton of work to this project. He’s responsible for the gemification, the new Flog reports, and the MD5 hashing. And David Chelimsky was nice enough to come over and personally show me around git and GitHub.

Try it out with:

sudo gem install jscruggs-metric_fu -s http://gems.github.com

Lone Star Ruby Conf Second Day (morning edition)

Sat, 06 Sep 2008 17:51:00 +0000

While checking out at the hotel I ran into Coby from confreaks.com They are recording the conference so you’ll be able to see the talks online sometime soon. They’re not cheap, but their videos are very good (they capture the output of the presenters computer and display it side by side with video of the presentation – you seriously need to check out their site), and they offer a discount if the conference is willing to CreativeCommons license the talks.

Starting off the morning was “Ruby: A Year of Innovation” with Gregg Pollack & Jason Seifer. They had a lot to cover so it went pretty fast. I tried to keep up as best I could.

Jason Seifer was up first:

HPricot - awesome for scraping websites
Juggernaut - server push with rails – keeps a connection open
Ambition - write your SQL statements in Ruby
Prawn - Ruby based PDF generation
Capistrano
Ruby VMs
roo - spreadsheets Excel, google docs
Dtrace and Ruby – memory stack live
Skynet - map reduce for Ruby
Data Fabric - easily shard your db with rails
Merb - the non-opinionated framework
Mack - Framework
Sinatra - good for restful
Webby - good for static websites
Shoes - dsl for gui apps
Hackety Hack - coding for kids

Gregg Pollack took over and talked about:

Redmine - Trac replacement in Rails
Pool Party - cloud computing config and maintenance in Ruby
Rad - robot control
Adhearsion - Voip
Rack - if you have a Ruby process you want to run on Mongrel, Thin, or whatever you just write one handler for Rack.
Phusion Passenger - servers up anything that supports Rack (including Rails)
Github - source control where you can see what others have done with your code and merge it back in without the modifier having to submit a patch (as they often forget)
make_resourceful and resource_controller - are two ways to puts resource handling in your controller
Starling - lightweight reliable message passing
EventMachine - Executes processes in one thread instead of trying to slice everything up. Works great for short tasks.
NeverBlock - can run SQL in parallel
Screencasts - Railscasts, PeepCode, PragProgrammers, EnvyCast

“Using jQuery with Ruby Web Frameworks” with Yehuda Katz

Yehuda used screwunit (a BDD javascript test framework) to BDD an implementation of a live updater in JQuery (which is an alternate Javascript framework). He had trouble with getting his tests to fail (some sort of caching problem). That’s rough in front of a 100 people. JQuery and screwunit look interesting, but I found it hard to focus on what he was doing ignore the difficulties of getting specs to pass. He finished his presentation with a declaration that you should never attempt to do a live coding presentation.

I thought it might be nice to step out of my comfort zone and take a look at “Resource-Driven Web Development With Waves” presented by Dan Yoder. Alternate title: “When Functional Programming Meets HTTP”

So, why use waves? Well, you can have more control over your app, use functional programming, and be very RESTful. Waves has Functor as much of its processing. It helps matching requests to response actions. You can have some pretty spare code that defines a bunch of restful paths.

Things that don’t fit anyone else:

I heard that there’s such a thing as Chicken Fried Bacon. Where the hell has this been all my life?
Joe O’Brien’s wife likes to say “Holy wow”
It was freezing in the conference center and 97 degrees outside. This is very Texas.
There was this clearing throat guy who would loudly clear his throat during every presentation. It totally caught me off guard when I was talking as I thought I had just said something terribly stupid. After my talk I noticed all the other presenters had the same reaction.
Where are all the ThoughtWorkers? I saw lots of ex-ThoughtWorkers but no current ones. All the cool kids must have gone to Rails Conf Europe, I suppose.

And now for Lunch. Which is Chicken fried Chicken.

Lone Star Ruby Conf First Day

Sat, 06 Sep 2008 04:32:00 +0000

(or the second day if you count the tutorials)

To start things off Jim Freeze got up and announced that there are 282 attendees and seats for 280 – So make friends. The the first talk of the day was “The Next Ruby” by Bruce Williams and he (of course) discussed the difference between Ruby 1.8.6 and 1.9

Bruce recommended a good test suite if you’re planning to move to 1.9 and I have to agree. Tests are good for lots of reasons (including design) but they pay for themselves 100 times over when you need to do a big tech migration.

Because of changes to string, anything string heavy (such as parsers) is gonna need some love. On the plus side:
“ruby”[0] => 114 in Ruby 1.8.6
“ruby”[0] => “r” in Ruby 1.9
Yep, I’ve been burned by that and I’m glad to see sanity carry the day.

In other news:

Hash.select now returns a hash
Hashes will maintain order
:this === “this” #true
Because of Multinationalization Strings now have each_chr or each_line not each (gotta support those Japanese, Chinese, etc. characters)
Threads are moving to native threading model
Fibers introduced: lightweight user threads (Check out NeverBlock)

But there’s a Huge Problem:
There exists a monstrous pile of unmaintained gems out there that everyone uses which need to be migrated to 1.9. So how are we going to get there? Bruce didn’t have any solutions. Last night at dinner Dave Thomas suggested that Rails will need to have some sort of killer feature that relies on Ruby 1.9 to get people to switch. He might just be right.

The next talk was “Hidden Gems” by James Edward Gray II and it featured lots of Battlestar Galactica photos whilst James talked about some of his favorite underused Ruby Gems.

NArray is high on his list because it borrows C’s numbers for speed. He had an array, used for an image, which was taking a few seconds to compute. When he switched to NArray he got hundredth of a second times. Nice.

James also likes SQLite for data processing. It’s surprisingly fast to load a bunch of data into SQLite and then use db power to search and process said data. You can even run it in-memory for even faster results.

And of course he talked about Rinda. Rinda is a fairly easy way to farm your tasks out to many different computers. And we all like more power.

Rein Henrichs then talked about “Ruby Best Practice Patterns.” He started out with a talk about how to De-factor your code for increased job security. Pretty funny stuff. He did some line by line examples of how to make code worse (example: use Pig Latin to name your methods). Then he segued into his real talk about how to write good Ruby code. His main point seemed to be that patterns are not just huge things that define architecture, they are an everyday practice. I totally agree with that and it’s kinda of cool that I get to follow his talk. I sort of pick up where he left off.

Then was lunch and a bunch of me getting nervous while looking over my slides. I had already given my talk in front of two different small groups of programmers, so I felt pretty good about it in the rational part of my brain. However, I had been scheduled to talk in the big room on the same stage Matz himself would later occupy – which is a bit of pressure. About a half hour before my talk I ran into Gregg Pollack and he shot a video of me summing up my presentation in 30 seconds. I was a bit keyed up so I have no idea how that came out. But now I get to be a part of his “#{whatever} Conference in #{some} minutes” series – a series I really enjoy. A few minutes before I was to get up and talk I heard some guys right next to me talking about how they were gonna ditch this metrics talk to go hear about javascript. Damn their black hearts.

I was pretty keyed up and the body mic was behaving weirdly, but I settled in after a few slides and I feel it went fairly well.

Next up in the main room was “Care and Feeding of Ruby Developers” by Steve Sanderson.
Steve comes from old school programming culture so this talk started off with some observations on these crazy young developers:

Aesthetics matter
Collaboration is important
Physical location is not that important (I disagree)
The doing is close to the planning
Programmer as Stars
Maybe you give away the code, but that doesn’t mean you don’t make money

And his advice for businesses employing Ruby devs was:

Be out there in the community
Empower the developers to make changes or at least have access to those who can make changes
Plan for people who blend technical and aesthetics

After that I headed over to the alternate room for “Creating Desktop Applications With Ruby on Mac OS X” by Brian Cooke

He started out by extolling the virtues of Cocoa. He loves and wants to marry Cocoa because it lets you create commercial grade apps without a ton of training. Then he talked about using Xcode and Interface Builder with Ruby Cocoa. Ruby Cocoa is a way to write slick looking Macish apps without having to learn Objective C. However the syntax looks a bit unlike Ruby because it has to be converted into Objective C.

It’s hard for me to follow these presentations about Interface Builder because the Object-Cish syntax gets in the way. Also the drag and drop stuff kinda freaks me out. To wire up a button you drag a blue cube somewhere, tell it what class to use, then you drag a blue line to somewhere else. Does that skeeve anyone else out?

He used the Faker gem to generate fake names. I saw Gregg use it in his ActiveRecord talk and I like the idea of having real-looking data in my tests. I’ll have to check that out.

Brian finished up by saying you really need to learn Objective-C because Ruby Cocoa is not fast, its files are available to anyone who gets the app, and debugging is no good.

Huh.

He then went on to say that MacRuby is going to be much better because it’s Ruby in the OSX runtime (kinda like JRuby). So you should get all the benefits of writing in Objective-C. When will it be out? No one knows ‘cause Apple is writing it and they’re all about the secrets.

Hey did I mention there was a “green room” for speakers? And that when I went in there no one yelled at me and told me to get out? Excellent. Also they had Coke Zero instead of the oppressive ice tea or water choices everywhere else. Although I do have to say that the food was pretty good. Excellent even, if you consider that it was conference food. I just don’t like ice tea.

I returned to the main room for “Ruby in the Cloud” by Mike Subelsky:
Mike had his start-up running on a single server when he realized he was running out of space. Faced with the choice of more servers, managed hosting, or the new and exciting “cloud” of EC2 he chose the cloud. Mike claims that it only took a few hours to build the server images and then only 20 minutes to migrate the code to be able to run all cloudy style. In case you don’t know, EC2 is a service offered by Amazon that lets you rent servers by the hour. You create a virtual image and fire one up as needed. They can bust at any time, and won’t be repaired, so plan for redundancy. Why would anyone put up with that? Because it’s crazy cheap and very scalable (Need more servers? Just fire up more and get a bill)

They made decision to use Ruby for all their server management. They used: Capistrano, Rufus-scheduler, God, FileUtils, Rush, Rake and a bunch of plain old ruby scripts.

Mike spent some time talking about how cool and readable rufus-scheduler can make your scheduling. Think of the last cron you wrote and compare it to this:

scheduler.every “10m10s” do
#stuff
end

Nice.

The big win of using Ruby everywhere is decreased context switching, leveraging of knowledge in Ruby, and OS portability.

Dinner was brisket. This would be the second night in a row I’ve had brisket. With any luck I can go 3 for 3.

The first keynote was given by Evan Phoenix and it was about memes in the Ruby community. Apparently, the Ruby community loves a good meme.

Dependency injection was a rash in 2004 caused by Java developers. DI wasn’t needed, sez Evan, because of the very nature of Ruby (as you can define stuff as late as you like).

Another meme: What’s this called:
class « self; self; end;
metaclass, singleton-class, or eigenclass?

Then he discussed the Singleton Pattern which he claimed was almost always a bad idea (more blaming of the Java folks for bring it up) and always bad for testing

Aspect Oriented Programming was a big topic for awhile. (He said that alias_method_chain is all the AOP you really need)

Evan discussed _why the lucky stiff as a meme. There’s so many rumors about _why. Who is he and what he does he do when he’s not dropping off excellent Ruby code in the middle of the night?

“Chunky Bacon” is a meme in its own right.

Here’s a somewhat exhaustive list of other memes he talked about:

Zed Shaw
git
Ruby is too slow
Macintoshes
TextMate
Rails can’t scale
Pixaxe
Ruby CPAN
rubygems vs rpa
ARGV parsers
DSLs
BDD
RSpec Backlash
Metaprogramming is cool

Evan’s take on all this was that the Ruby community loves to have fun. It’s a very light on its feet community. This can scare a lot of enterprisey people, but he finds it awesome.

The second keynote of the night was given by Matz on Ruby’s “Past, present, future”

I always find it hard to sum up Matz’s presentations. They are funny, inspirational, and touching in a way that doesn’t lend it self to summary. You come away with the feeling that this man really wants to write a language that will make you a happier person.

Topics he discussed include:

Why Ruby is simple, but not too simple (or Ruby vs Lisp)
Interpreters
1.9 features
2.0 notions
The primary goal of Ruby is to enjoy programming

Then he graciously answered questions for 30 minutes. And got a standing ovation.

Training Day at Lone Star Ruby Conf

Fri, 05 Sep 2008 03:49:00 +0000

First, I’d to extend a big thanks to Joe and Jim for giving me a ride to the conference today.

The two tutorials I decided to attend today were:
“The Advanced ActiveRecord Workshop” with Gregg Pollack & Jason Seifer (the Rails Envy Guys) and “The Ins and Outs of Ruby I/O” with James Edward Gray II and Gregory Brown. Both of which were excellent.

The Advanced ActiveRecord Workshop covered:

Loading large data sets (the ‘ar-extensions’ gem helps lots).
A gentle reminder to properly index your freakin’ database.
How Rails optimizes multiple :includes in finds and how that goes to hell if you combine lots of :includes with :conditions
The super awesome Named Scopes (which you need to look up right now, if you haven’t already because it’s not just about defining custom finders – it’s about defining custom finders that you can chain together and get optimized queries)
Polymorphic Associations, Single Table Inheritance, Dirty Fields, Association Proxies, Callbacks, and creating AR plugins.

I was really impressed by the smoothness of their presentation. It was a great balance of funny/informative/challenges. The challenges were a good way to make the audience think about what had just been presented. If I had one criticism it would be that the later challenges seemed too hard for the time allowed. I found that if I made one stupid mistake (which I then had to track down) I would not finish by the time they gave out the answers. I suspect they were rushing to finish on time. Overall it was great stuff and they say that this talk was an elongated version of their recent EnvyCast on ActiveRecord so I would say that it is definitely worth your 9 bucks.

The Ins and Outs of Ruby I/O was also very informative. Since Ruby has such easy file IO it’s easy to forget about the underlining structures and the problems that can crop up. Topics include:

The most efficient way to read in a file (generally foreach)
Random access inside a file
Overcoming buffering problems with file.fsync file.flush (fsync returns nil if it doesn’t work)
A resonably through regex explanation
A compare and contrast of how to handle unicode on Ruby 1.8 vs Ruby 1.9
How to use Marshall or Yaml to serialize your objects.

The talk was being video taped so I assume it will show up on confreaks which, btw, is my new favorite site. You can see the whole freakin Ruby Hoedown on there. And you really should check out Rick Bradley’s “flog « Test.new” talk. He takes Flog, forks it into a new project called Flame, puts it under spec, and refactors it to be less complex (nice bit O’ irony there).

Later we had some excellent Bar-B-Q at Rudy’s and discussed how the hell the Ruby world is going to migrate from 1.8.x to 1.9. We had no good solutions. The Bar-B-Q was excellent, however, so we left in good spirits.

Tomorrow I’ll be speaking at 1:30 on “Using Metrics to Take a Hard Look at Your Code” so I’m pretty excited as this will be the first mass presentation of this talk and I think its turned out nicely.

I'll be Presenting at Windy City Rails

Tue, 26 Aug 2008 01:01:00 +0000

If you’re going to be in the Chicago area on or about Saturday, September 20th, you can see me give my “Using Metrics to take a Hard Look at Your Code” talk at the Windy City Rails Conference. David Heinemeier Hansson, David Chelimsky, and Noel Rappin have been announced as speakers and I’ll be joining them for a interesting day of Rails related goodness.

Here’s the abstract:

It’s an interesting fact of human nature that you can’t do something every day and not secretly suspect that you’re good at it. Which goes a long way toward explaining why everyone thinks they write fine code. To combat this self-delusion you can use metrics to take a hard look at your application. This talk will discuss the ways in which you can measure how good your Rails project really is. And how to fix the bad parts.

Using a daily metrics build (run every day by CruiseControl.rb) you can compile a ‘hit-list’ of the worst methods in your application. Then your team can spend some time every iteration trying to refactor these methods.
In my talk I will discuss 4 tools to identify problems:

* Code coverage with Rcov
* Cyclomatic complexity with Saikuro
* Flogging with Flog
* Using source control to figure which files change the most

All of which are pretty easy to use, especially if you use metric_fu – a Rails plugin (blatant plug). Once you’ve figure out where the badness is, you must fight it. While showing examples, I’ll be discussing:

* Why 90% (or even 100%) test coverage isn’t a silver bullet against bugs.
* How Saikuro and Flog measure complexity and the advantages of each.
* When complexity is OK
* Why ‘inject’ sucks, except when it’s pretty cool (see above point).
* Strategies for figuring out if your fancy metaprogramming is worth the extra complexity and what to do if it isn’t.
* What is good enough coverage.
* Failing your build on poor metrics numbers.
* Why you should always remember Carlin’s law:
“Have you ever noticed that anybody driving slower than you is an idiot,
and anyone going faster than you is a maniac?”

I'm Thinking of Putting View Logic into a Model

Mon, 25 Aug 2008 01:43:00 +0000

As I am clearly crazy, but hear me out. Say there’s a Car object that uses single table inheritance and all the objects that descended from it are routed through the car method in the CarController. The CarController looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

class CarController < ApplicationController
def car
layout = nil
@scripts = []
@stylesheets = []
@car = Car.find(params[:id])
if @car.is_a?(Toyota)
layout = ‘two_column’
@stylesheets « ‘two_column_layout.css’
@stylesheets « ‘two_column_theme.css’
elsif @car.is_a?(Hyundai)
layout = ‘three_column’
@stylesheets « ‘three_column_layout.css’
@stylesheets « ‘three_column_theme.css’
@stylesheets « ‘hyundai_theme.css’
@scripts « ‘discount.js’
elsif @car.is_a?(Ford)
layout = ‘ford’
@stylesheets « ‘ford_layout.css’
@stylesheets « ‘ford_theme.css’
@scripts « ‘discount.js’
elsif @car.is_a?(Saturn)
layout = ‘two_column’
@stylesheets « ‘two_column_layout.css’
@stylesheets « ‘two_column_theme.css’
@scripts « ‘poll.js’
#
# 100 more lines like above
#
end
render :action => find_view, :layout => layout
end
end

So I’m not very happy about this and ultimately I’d like to refactor the app to use different controllers for different objects (crazy huh?), but that is a major undertaking as the whole app assumes this architecture. It occurred to me the other day that there’s an in-between step. I could do something like this:

1
2
3
4
5
6
7
8
9
10
11

class Car < ActiveRecord::Base
def layout
‘two_column’
end
def stylesheets
[‘two_column_layout.css’, ‘two_column_theme.css’]
end
def scripts
[]
end
end

And then the Ford class would look like this:

1
2
3
4
5
6
7
8
9
10
11

class Ford < Car
def layout
‘ford’
end
def stylesheets
[‘ford_layout.css’, ‘ford_theme.css’]
end
def scripts
[‘discount.js’]
end
end

So the CarController could then be shortened to this:

1
2
3
4
5
6
7
8
9

class CarController < ApplicationController
def car
@car = Car.find(params[:id])
@scripts = @car.scripts
@stylesheets = @car.stylesheets

render :action =\> find_view, :layout =\> @car.layout     end   end

Any model that needs to overwrite the default layout, scripts, or stylesheets can.

So should I take this step and put view information in the models, or should I just hold out for the big refactor into separate controllers? Keep in mind the big refactor is so big that it may never happen.

Perks vs. Salary

Fri, 22 Aug 2008 02:52:00 +0000

When I worked for ThoughtWorks I once got on a bus after work – no ordinary bus, mind you, this was “Party Bus.” On this “Party Bus” there were leather seats, flashing lights, a big screen TV with karaoke, and many bottles of Patron Silver and Grey Goose. We drank and sang like maniacs, then we got to the Casino where we ate a very fancy dinner (including some very pricey wine), did a little gambling, caused some trouble, and then back on the bus for more drinking and karaoke. The only thing I payed for was the gambling.

The next day I was pretty hung over but I still got to pair with some of the best programmers I’ve ever worked with.

I often think about this when people ask how ThoughtWorks is able to keep such excellent programmers when they could easily go somewhere else and make more money. If you took all that money that was spent on our night out, divided it by the 10 or so people that went, and added it to our salary, there’s no way we would have valued it anywhere near as much. I now have a story I can tell for the rest of my life and that’s worth something.

Which is why I was pretty happy when Obtiva (my new employer) took all of its employees and their wives, girlfriends, and children out for a ‘booze cruise’ (with good food too) on Lake Michigan during the Chicago Air and Water Show last weekend. Perks inspire loyalty and camaraderie far beyond their monetary value.

Not having to get on a plane twice a week is nice too.

What's a Good Flog Score?

Thu, 21 Aug 2008 01:44:00 +0000

I’ve been using Flog to measure the complexity of my Ruby code and I was wondering how others interpret Flog numbers. I’ll go first with my opinions:

Score of Means
0-10 Awesome
11-20 Good enough
21-40 Might need refactoring
41-60 Possible to justify
61-100 Danger
100-200 Whoop, whoop, whoop
200 + Someone please think of the children

(note: these are scores for an individual method)

The 20 - 60 range is interesting to me. Sometimes the complexity of what you’re doing will justify the score and other times the method is in desperate need of refactoring.

The highest Flog score I’ve seen in the wild was in the 300’s. What’s you’re personal high and what do you think of my rankings?

In other news, I’ve redesigned the look of my blog so if you only see my post through a reader you might want to stop on by, take a look, and tell me what you think.

Subversion Password Caching Problem Leads to TextMate svn Integration Problems

Tue, 05 Aug 2008 04:13:00 +0000

I had a problem recently with TextMate’s svn integration and finding the solution taught me some interesting things about Subversion and TextMate that I’d thought I’d share:

First – TextMate fails with a non-helpful error message if you don’t cache svn passwords. For a long time I didn’t even realize that my two problems were related. I had to type in my username and password for svn every-time (even though I wasn’t using svn+ssh). And TextMate exploded when I tried to see the svn log of a file. Turns out one causes the other – obvious in retrospect.

Second – Subversion keeps passwords and configuration files inside a .subversion folder inside the user’s home directory. Try looking for a strangely named file inside ~/.subversion/auth/svn.simple for cached passwords. And there’s a config file at ~/.subversion/config If you want to stop this caching of passwords you can set:
store-passwords = no
inside this file.

Third – Subversion will fail to cache silently if the permissions are not right on the folder where it attempts to save the username/password. This was the sticking point for me – everything I tried failed and I wasn’t getting any helpful error messages (and none from the command line svn client) but then I noticed that my ~/.subversion/auth/svn.simple directory had different permissions (and a different modified time) than the other folders next to it. One simple chmod command later and I was caching passwords and using TextMate’s svn integration like nobody’s business.

Fourth – Yes I still use Subversion – feel free to leave me a comment telling how much happier I’d be with git.

Resources:
Subversion and TextMate
Short version of Subversion’s password caching
Longer version

Presenting at Lone Star Ruby Conf

Tue, 01 Jul 2008 02:35:00 +0000

Word on the street is that Lone Star Ruby Conf was one of the best regional conferences last year, so I’m pretty excited to be presenting at the 2008 edition. It’s happening Sept 4-6th in Austin, TX and speakers include: Matz, Evan Phoenix, Jim Weirich, The Rails Envy Guys, and too many other luminaries to name.

I’ll be giving a talk entitled “Using Metrics to Take a Hard Look at Your Code” which is a distilation of my thinking over the past year about how to use numbers not as a hammer with which to beat people, but as a tool to improve your code.

Registration just opened up today, but space is limited so go sign up soon.

Disconnecting RSpec from the Database

Wed, 18 Jun 2008 04:41:00 +0000

Recently I spent some time away from RSpec and I did miss it, but one thing especially liked about the testing framework I lived in for 10 months was that it had two ways to test anything you liked: One that hit the database and one that did not. We used UnitRecord to disconnect unit tests from the database (I should mention that we defined unit tests as tests that didn’t hit the db and functional tests as those that did). So the vast majority of our tests didn’t really need to hit the db and ran very fast. Sometimes we wanted or needed to hit the db and we did (only occasionally in model tests, more often in controller tests, and we kept logic out of the view and only tested views at the Selenium level). It was pretty cool and I’d like to thank the team who set it up before I got there.

There’s been a lot of talk about whether model, view, or controller tests should hit the db but I’ve found that there’s enough exceptions that it’s worth just saying that you should try not to hit the db when possible for speed’s sake and then when the test becomes 20 lines long because of all the mocking/stubbing required, go ahead and hit it. That’s all well and good, but how do you disconnect the db from an RSpec example?

I, personally, have had success with NullDB. NullDB is the null object pattern applied to databases. Which is a fancy way of saying that it swallows up any call that would go to the db and returns a nil. So you mock/stub all you like, but if you forget or just don’t care about a particular call then don’t worry because nothing goes to the db and nil is returned (which, surprisingly, is more than enough in most situations). On the NullDB page I’ve linked to, the author provides a way to universally disconnect RSpec from the db, but, as I’ve already said, I’m a bit pragmatic when it comes to disconnection so In my spec_helper.rb I define something like this:

1 share_as :Disconnected do
2
3 before :all do
4 ActiveRecord::Base.establish_connection(:adapter => :nulldb)
5 end
6
7 after :all do
8 ActiveRecord::Base.establish_connection(:test)
9 end
10 end

and then in any “Describe” block where I’d like to disconnect the db I do this:

1 describe User do
2 include Disconnected
3 # ‘it’ blocks or
4 # more describes
5 end

If I want to hit the db I just don’t include Disconnected.

Now since NullDB fails silently I’ve heard tell of problems with it being hard to debug (although I have yet to experience these problems). Any spec/test/example that just isn’t behaving properly can be exiled to the ‘hit the database’ area. Also I should mention that when I tried to use NullDB with Rails 1.2.6 I got a bunch of errors but it worked fine with 2.0.2. If you are looking for a more vigorous enforcement of ‘don’t touch the db in unit tests’ then UnitRecord may be for you. I thought RSpec and UnitRecord didn’t play nice, but my fellow Obtivian, Ryan Platte informs me that he’s got it working. So I’m calling him out to post about disconnecting the RSpec with UnitRecord.

Breadcrumbs are Evil

Fri, 13 Jun 2008 01:36:00 +0000

I’ve been on a few projects where I’ve had to mess with breadcrumbs and they tend to be much more trouble than they’re worth. It all starts simply enough: put the users path through the app on the page. Which isn’t too hard.

Then the bugs come.

Like: “This page is linked to from page A but it really is more of a child of page B so the breadcrumbs should show page B and not A even if they come from A”

Or: “When I come to page X from the search on the front page it should have breadcrumbs with Y and Z because that’s really what it’s under.”

And: “Page L is listed under K but should be under M (followed a few days later by) Page L is listed under M but should be under K”

Oh, you want a hierarchy. Which is hard.

First: There’s now a new job to be filled: Taxonomist. Somebody is going to have to figure out where all the pages in the app fit in some sort of tree. And this list must be maintained.

Second: There must be code that manages all this. And since there are always exceptions to the rules of hierarchy it can get quite complicated.

Third: Although your customers don’t really care where the page fits in the hierarchy, everyone else in the business does. A lot. So much so that there will be many (perhaps endless) arguments about exactly where things “belong.”

All this will take plenty of time that could be used doing something that, you know, your customers may actually need. I guess I really don’t have too much problem with “true” breadcrumbs (aside from the fact that I never use them and they’re kind of a relic from the days of “Wow the internet is so complicated people will get lost!” and they can probably be replaced with a good search box) but what I really hate is when they act as a gateway drug to hierarchy.

Syntax Highlighting for Ruby Made Very Easy

Thu, 29 May 2008 22:00:00 +0000

So I’ve been making do with lame-o ‘pre’ tags for my code snippets and my blog looks, well, lame. But the various alternatives for highlighting ruby either didn’t work for Blogger or smacked of effort (and you know how I feel about effort).

Well, recently I joined Obtiva and I was being all nosy looking at my new co-worker’s blogs, when I found Tyler Jennings post about creating Spotlight. Take some ruby code, paste it into the text box, press a button, and you get some html which you copy and past into your blog. You’ll have to include the default css in your css file the first time, but that’s not too hard. Output looks like this:

1 class Echo < ActionMailer::Base
2 def copy formatted_message, incident_id
3 users_with_echo_emails = User.find(:all,
4 :conditions => “echo_email is not NULL”)
5 users_with_echo_emails.reject! {|user| !user.receive_echo?}
6 recipients_array = users_with_echo_emails.map { |user| user.echo_email }
7 recipients_array « ENV[‘default_echo_email’] if ENV[‘default_echo_email’]
8
9 recipients recipients_array
10 subject “Globex pager incident #{incident_id} updated”
11 from ‘noreply@globex.com’
12 body :message => formatted_message, :incident_id => incident_id
13 end
14 end

Cool huh?

Saving in TextMate

Wed, 28 May 2008 00:22:00 +0000

So today, for something like the fifth time, I sat down in front of a TextMate installation that didn’t have saving set up the way I like it: Auto save on loss of focus and when running a test (or spec). And, for the fifth time, I had to dig around the internet for the answer. So, as a public service to my future self:

Save files when focus is lost:
http://macromates.com/textmate/manual/saving_files
TextMate –> Preferences –> Advanced –> Saving

Save all files when running a test (or spec):
http://macromates.com/textmate/manual/commands
for specs:
Bundles –> Bundle Editor –> Edit Commands –> Rspec –> Run Single Example
Bundles –> Bundle Editor –> Edit Commands –> Rspec –> Run Examples
for tests:
Bundles –> Bundle Editor –> Edit Commands –> Ruby –> Run
Bundles –> Bundle Editor –> Edit Commands –> Rspec –> Run Focused Unit Test

Change the save drop down to “All Files in Project”

Hope this helps, future Jake.

Metric_fu Now Measures Churn

Mon, 12 May 2008 02:39:00 +0000

Sometimes a class does too much. It’s used everywhere, does everything, knows all about the nasty internals of all the other classes, and every time you change anything in the application it has to change. Or maybe you have a class that everyone loves to refactor because it’s so bad. One developer changes it to be better, then another changes it to be a different kind of better, and this keeps happening without it ever getting any easier to use. There’s a lot of bad reasons why a class might change all the time so I added a churn report capability to metric_fu version 0.6.0.

Now you can use metric_fu to create a report like this:

By issuing this command:

    rake metrics:churn

And use the results to find classes that might have problems. It uses ‘svn log’ to create this report so currently you need to be using subversion. (If anyone out there want to contribute some code to get it working with git, lemme know.) The default is to start counting changes from the start of the repository’s creation. That may be too much for your project so you can change the start by adding these lines to your Rakefile:

    namespace :metrics do
     CHURN_OPTIONS = { :start_date => lambda { 3.months.ago } }
   end

The Proc is there because ‘3.months.ago’ only works after the Rails Environment is loaded (and Rails extends Fixnum) which I didn’t want to do every time you run a rake task.

You can also change the minimum churn count (files with changes less than this number won’t be included in the report) like so:

    namespace :metrics do
     CHURN_OPTIONS = { :minimum_churn_count => 3 }
   end

Also in this release I added the ‘metrics:all_with_migrate’ task because of some feedback I got that running ‘metrics:all’ in a cc.rb build fails if there have been migrations. So the new task migrates the test db before running all the metrics tasks.

Check out the metric_fu project home page for more information and installation instructions. Or you can jump right in by doing this in the root of your project:

script/plugin install \
 http://metric-fu.rubyforge.org/svn/tags/CURRENT/metric_fu/
sudo gem install rcov
sudo gem install flog

Dead Simple Rails Metrics with metric_fu

Sun, 27 Apr 2008 22:45:00 +0000

Every time I create a new rails project I usually put off writing tasks to analyze the code’s quality ‘cause it takes time and time is, you know, finite. So I’ve decided to extract some code into a rails plugin which I call metric_fu.

It’s a bunch of rake tasks that produce reports on code coverage (using Rcov), cyclomatic complexity (using Saikuro), flog scores (using Flog), and rails stats (using ‘rake stats’). It knows if it’s being run inside a CruiseControl.rb build and puts the output in the Custom Build Artifacts folder so when you view a build you see this:

The coverage report is your standard rcov report:

Flog output is thrown into an html file:

At the end metric_fu calculates the average flog score per method: You might want to check out my previous posts on what to do with a Flog report: The Method Hit List and When You Should Ignore Metrics

Saikuro’s output is the same as always: (I changed the warning and error levels for this pic – more on how I did that later)

And ‘rake stats’ is always useful:

So how do you get all these reports? 1. install Flog sudo gem install flog

2. install rcov sudo gem install rcov

3. install metric_fu ruby script/plugin install
http://metric-fu.rubyforge.org/svn/tags/REL_0_5_1/metric_fu/ (in the base of your rails app)

4. rake metrics:all

Which should work fine if you have standard Rails testing and you like my defaults. But what if you use a combination of RSpec and stock Rails testing? Then you can insert this into your Rakefile:

namespace :metrics do TEST_PATHS_FOR_RCOV = [‘spec/*/*_spec.rb’, ‘test/**/_test.rb’] end

The namespace isn’t strictly necessary, but I like it for intentional purposes. Multiple paths are useful if, like on my last project, you need to be specific about which tests to run as some tests go after external services (and the people who manage them get cranky if you hammer ‘em a lot).

If you also want Rcov to sort by lines of code (loc) and have more aggressive cyclomatic complexity settings then do this:

namespace :metrics do TEST_PATHS_FOR_RCOV = [‘spec/*/*_spec.rb’, ‘test/**/_test.rb’] RCOV_OPTIONS = { “–sort” => “loc” } SAIKURO_OPTIONS = { “–warn_cyclo” => “3”, “–error_cyclo” => “4” } end

That’s it – hope you find it useful, lemme know if you find a bug, and check out the project home page at: http://metric-fu.rubyforge.org

Oh, and thanks to all my co-workers who helped write the original code, in its various forms, that became this plugin.

Update 9/22/2008 - metric_fu is now a gem, on GitHub, and is useful for any Ruby Project. Check the home page for current information.

The Bug that Took Down a Release and How I Wrote It

Wed, 16 Apr 2008 03:25:00 +0000

We were disposing of some last minute bugs today and I was reminded of the time, a few projects ago, when I introduced a bug that screwed up a whole release.

Let’s say that I was writing a pizza ordering web site and, after many successful releases, the Big Wigs at PizzaCo wanted to introduce a new feature that would let customers customize their pizzas even more. In production the user could select style of pizza (thin, pan, stuffed), size, and toppings. There was even some cool logic in place to make sure the toppings were available before being offered. Now, in this brave new world of pizza, the customer would be given the chance to customize each topping. If you chose peppers, you would be given the choice of green, yellow, or red (through the magic of javascript). A choice of onions would prompt the question of red or yellow. Pepperoni? – regular or spicy?

So we wrote the feature, which was difficult, as we had to totally change how we interacted with the Topping Provider Service (TPS) in order to reserve these new toppings and check for availability. But then the Big Bosses had a thought: “What if this new extra bit of choice scares away the customer?” And their solution was to have “switch” to turn the extra customization on and off. As developers we pointed out that this would be costly as making the new system look like the old, but still work with the new under the covers (by selecting defaults for each topping), was not easy. And it would introduce extra complexity by having two different pages that do almost the same thing. But they wanted it, so we did it.

A few weeks later we found a bug with the topping selection page: If you take the last bit of pepperoni, but later come back to the toppings page to change your selections, the TPS service thinks all the pepperoni is gone (because you just reserved it) so you don’t see that option on the page. We fixed it by looking at the customer’s saved pizza and putting any toppings missing from the page back on the page. Except that we forgot to make the changes in the page that mimics the old behavior. Even worse, the way we fixed the new toppings page caused the pseudo-old page to blow up if you ever try to go back to it. And we found out that this happens right before a release.

And this was all my fault.

I worked on the feature to add the new customization, I also worked on the feature to hide the new feature, and (believe it or not) I worked on the bug fix. There was no one else on the team who was in a better position to remember that there were two pages and each need a change. Aside from my obvious point that I’m a terrible developer and you should never hire me, I think there’s an interesting lesson here about unused functionality.

We wrote the new page and then we used it. It did all sorts of sexy Ajax and was cool. And after a few weeks we forgot about the legacy page and things that might break it. There are many times when it will be tempting to bifurcate your code, but every time you do so you create a place for a bug. This is especially true if one path is rarely used. The new code will continue to evolve and the old will sit around and collect bugs. We did a full pass through regression testing, in addition to our normal tests and QA, but we didn’t find this bug until hours before the release. Which is better that finding it after, but still, if the release had gone out it would have had the new feature turned off (for “safety”) and the monster bug on. We were mere hours away from disaster.

I know it’s hard not to want options, but every option has its price.

Ruby Substrings and Testing Legacy Code

Tue, 01 Apr 2008 03:11:00 +0000

Recently Josh Cronemeyer and I were working on writing a game in Ruby. Gosu, a 2D game library for Ruby and C++, and Chipmunk, a 2D Physics engine, do lots of the heavy lifting so we thought it would be a fun Saturday afternoon thing to do. However, the examples had no tests so when we tried to change some stuff, and of course it didn’t work, we were all sorts of clueless as to why.

Well, wrapping tests around legacy code is not a lot of fun but it is an effective way of debugging. Here’s how I like to do it:
Step one:
Identify potentially troublesome code (As in “What the hell is that doing?”).
Step two:
Write some tests that verify the functionality of the code.
Step three:
Pull out the offending code into a method or methods.

There’s a fair bit of interplay between steps two and three, testing one piece may require extracting it.

Step four:
Either you’ve found your problem or you’ve accomplished four things:
1. You really understand the troublesome code.
2. You’ve refactored it into a more readable version.
3. You’ve put some code under test.
4. You at least know where the problem isn’t.

I adopted this strategy after reading Michael Feathers “Working Effectively with Legacy Code” -- which is good reading.

Eventually we found the problem to be an erroneous substring.
“hello”[0] doesn’t return “h”
Ex:

irb(main):001:0> “hello”[0]
=> 104

Which is the character code of ‘h’, so we tried:

irb(main):002:0> “hello”[0,1]
=> “h”
irb(main):003:0> “hello”[4,5]
=> “o”

Aha! Now we thought that “string”[x,y] gives you the characters between position x and y. But no, we got some really strange behavior until we realized that “string”[x,y] gives you y number of characters starting at x position. The reason “hello”[4,5] gives “o” is because if you ask for 5 things starting at position 4 there’s only one character left, which Ruby is only too happy to hand over. “hello”[0,1] is the way to get the first letter, just as “hello”[1,1] is how to get the second letter.

Now one of the things that drew me to Ruby in the first place was it’s intuitiveness. Often I found that if I didn’t know how something worked, I could just write what I thought would work and it did. But substrings are a rare anomaly – I find I have to look up how they work every time. And I still make mistakes even after I check the ruby doc. Does anyone know the history behind this weird syntax?

On Inject, Complexity, and George Carlin

Wed, 19 Mar 2008 02:33:00 +0000

Inject has been stalking me for over 2 years.

The year was 2005 and I had decided that I would write a website in this new fangled Rails thing. But while debugging some plugin code I ran into inject. To my java encrusted eyes, it looked like this:

weird_thing = dont_know.inject({}) {

m, o

m[o] = some_crazy_method o; m}

Holy crap. I had no freakin’ clue. I looked up the docs on inject and saw that inject, from what I could understand, was used to add up numbers in an array. Which confused me even more. Of course since then I’ve come to understand inject, but have remained wary as it can be awful confusing to those new to Ruby. Just recently I wrote about how some teammates and I decided against using it to shorten a method, because the particular use sacrificed readability. Then Brian Dainton wrote about how he loves inject. And finally, my team got in a rather spirited discussion over its use. One side of the argument was that it should be used with care as it’s hard to read, especially for Ruby newcomers (whom we will be handing this project over to sooner or later). And the other side pointed out, rightfully so, that the reason we use Ruby in the first place is precisely because of powerful methods like inject. If we have to stay away from inject, or method_missing, or metaprograming, or anything else that is deemed ‘too hard’ then why not just write in Java or C# or Visual Basic?

All this reminded me of one my favorite quotes form George Carlin:

Have you ever noticed that anybody driving slower than you is an idiot, and anyone going faster than you is a maniac?

The funny thing is that I can see both sides of this argument. Back in 2005 I would have thought you crazy to use such a method. Now it would take only a moment to figure out that in:

weird_thing = dont_know.inject({}) {

m, o

m[o] = some_crazy_method o; m}

dont_know is probably an array full of objects and the author of the code wants to pass each one of them into some_crazy_method, get the result, and then put that result in a hash with the original value as the key. When I was a slow driver in Ruby I thought all the other speed daemons were crazy to write such complex code. Now that I’m a budding speed racer myself, it’s temping to see people who can’t understand my code as stupid.

At this point it would be pretty easy to wrap up this post by saying something useless like: “Use your best judgment when adding complexity, making sure to balance blah, blah, blah…” You, like everyone else on the planet, think you know the difference between too slow (code for dummies) and too fast (code for computers only). Even if you’re only wrong 10% of the time (which I doubt) you’re going to write a lot of horrible code.

So how do you know when you’re wrong (or right)? You don’t. Let me try to explain though two examples of complexity: one good and one bad.

In our application, we have a number of different ways to go through a sign up process. So we created an object called Flow. And each flow contains a number of FlowPages that specify the order of the flow. With a little bit of Ruby magic, we made it so you can define a flow like this:

NormalFlow « FirstPage « SecondPage « ThirdPage

Which is damn cool. But there was a bunch of code needed to support this. Over time we came to realize that we weren’t seeing a lot of benefit from this notation. We didn’t define flows that often and even if we did, the much simpler:

NormalFlow.new(“FirstPage, SecondPage, ThirdPage”)

expresses the intent just as well. When the time came to play a story dealing with putting flows into the database so the business can try different paths, we saw that the complexity was actually making it harder to use and so we threw out the old code and went with the totally unsexy “pass in a string” constructor. Lame, but way easier to maintain.

In the other example, we had a requirement that the business should be able to change any of the words on the page quickly (without a release). This, of course, meant putting the page content into the database. We created a PageContent object that, through the magic of Ruby, gets created before any view is rendered, looks up all the content that it might need, and dynamically defines methods on itself that correspond to its contents. A given page may have a title or an address_line_one_label and to access these methods in the view you can call: content.title or content. address_line_one_label and you don’t have to do a thing to set this up (other that put content in the db) – it’s available in every view. And over the months since we implemented this feature it’s been easy to use, extend, and maintain.

And that’s really my point here. I can’t just say “use your best judgment” because you already are. Furthermore, when you’ve just learned about, say, metaprograming you’re probably going to use it more than you should. You won’t have any idea if that super cool bit of Ruby is good or bad complexity until you get back in there and try to make it do something new. If the complexity fights you at every turn, it’s bad and you should get rid of it. Good code you won’t even notice because the tasks that touch it go so smoothly.

A Small Rails Site and Why You Should Build One

Wed, 12 Mar 2008 02:45:00 +0000

The other day I was showing Rails off to a friend of mine who wanted to know what all the fuss was about and I realized a shocking thing: I was horrible at starting a Rails site. Why? Because for the last year or so I’ve been working on large, established Rails codebases. Which is kinda cool as it shows how far Rails has come, but it sucks because one of the very cool things about Rails is how much you can do in a short amount of time.
which has an admin section so he can upload pics and get them resized and thumbnailed with 10% of the effort.
but I was impressed how much better it’s gotten in the last year. You can now use ImageScience or MiniMagick as the back end image processor and you can save files to Amazon’s S3. And, maybe it’s just me, but it seems like it’s easier to use than ever. Restful_authentication was also impressive in its gains. I’ve had mixed results with Rails plugins for authentication in the past, but everything went very smoothly. One of the things I really liked was the ability to omit the action mailer part of the plugin. Sometimes you just don’t need to be able to email lost username/passwords and it’s nice to be able to leave that code out. Another nice surprise was after upgrading to the latest IntelliJ I found that creating a project with RSpec was an option. Worked well, too.
Rails website as you’ll have fun and learn a bunch of things your day job won’t teach you. Especially about deployment – but that’s another post.

ActiveSupport::CoreExtensions::String::Inflections underscore

Thu, 21 Feb 2008 12:26:00 +0000

> “ActiveSupport::CoreExtensions::String::Inflections”.underscore
=> “active_support/core_extensions/string/inflections”

> “ActiveSupport::CoreExtensions::String::Inflections”.underscore.camelize
=> “ActiveSupport::CoreExtensions::String::Inflections”

Looks like Rails extension to the string class does more than underscore a string. Which is useful, but shouldn’t the name be “relative_path” or something similar? When I see the method name “underscore” I know exactly what it means, except that’s not what it does. The opposite of “underscore” is “camelize “ which, again, may not have the best name.

And if you want to find the declared constant specified in a string:

> “ActiveSupport::CoreExtensions::String::Inflections”.constantize
=> ActiveSupport::CoreExtensions::String::Inflections

Useful and an excellent name.

Return to Generating a Unique Number

Wed, 20 Feb 2008 05:09:00 +0000

In a previous post I discussed generating a unique id number for an order in our database. Read the original post for the full story, but to sum up, I decided that playing around with rand(100_000_000) was good enough. Martin Ankerl posted a comment pointing out that I was vulnerable to the Birthday Problem. So after a quick trip to wikipedia, I found out that if you have 23 people in a room you might think you have a 23/365 chance of having 2 people with the same birthdays, but no, your chances are closer to 50%! This is because in a group of 23 people there are 253 pairs each of which have a 1/365ish chance to have the same birthday. The formula to calculate how many are needed to give you a 50% chance of having a birthday collision is:

.5 + sqrt(.25 + 2 * 365 * ln(2) ) = 22.9999

Which means the 50% chance of collision mark for 100,000,000 is:
.5 + sqrt(.25 + 2 * 100,000,000 * ln(2) ) = 11774.600235771

Holy crap! I’m screwed if I can only have 12,000 orders before I get a 50% chance of collision. But wait, I think I’m ok because every time I insert a key into the database I check to make sure it isn’t already in there. This means that when I’m at 12,000 orders the next key has only a 1 / (100,000,000 - 12,000) chance of being a collision. I think I’ve changed from the “Birthday problem” to the “Same birthday as you” problem, which is way more intuitive. If I’m wrong I rely on you, the blogoshpere, to tell me so.

If you haven’t already, go read the wikipedia article on the Birthday problem. It’s pretty cool.

When You Should Ignore Metrics

Wed, 23 Jan 2008 12:44:00 +0000

Our team has been doing a bang up job of reducing our complexity through our hit list (All of our methods are ranked by Flog score and we spend some part of every iteration picking the worst methods and trying to refactor them.). But sometimes we run into a situation where we prefer a high Flog score to a low one. For instance, this bit ‘o code:

def is_something?(x, y)
x.foo == y.foo &&
x.bar == y.bar &&
x.blah == y.blah &&
# and 10 more lines of the same
end

which had a Flog score in the 60s. We could have changed it to this:

def is_something?(x, y)
[“foo”, “bar”, “blah”, # and so on ].inject(true) do | a, b |
a && x.send(b) == y.send(b)
end
end

Which gave a us a much lower score. However that first method is crazy simple. Merely glance at it and you know what it’s doing. The ‘less complex’ method? Even an experienced ruby dev would need a moment or two. As for a new dev… I can remember a time not so long ago when I found an inject in some source code and was pretty confused even after I looked up the Rdoc. It’s one of those methods that’s so powerful and flexible that the documentation only seems to make sense after you understand it.

After some discussion with the team we decided that while Flog is very good at telling you which methods to inspect for badness, it is not always true that a better score makes a for better method. The same can be said for all metrics, really.

Automated Javascript Rails Testing

Tue, 22 Jan 2008 05:08:00 +0000

On my last project we did some javascript unit testing in browser using the unittest.js library from scriptaculous, but because it’s kind of a pain to set up and integrate into your build, I didn’t get it working on my current job. Which makes me a bad person. However, Dr Nic Williams has made yours and my life easier with his javascript_test plugin for rails. Now I can type:

ruby script/generate javascript_test fancy_javascript_file

And get a fancy_javascript_file_test.html to test my fancy_javascript_file.js. It’s all set up to go right at the javascript file. Just open the html file in a browser and it runs the tests. The sweet doctor explains it all better than me so go check out his site.

What I want to talk about is trying to integrate this into your Cruise Control build. The first problem we faced is that the tasks that come with the plugin don’t close the tabs created in the browser. So after 10-20 builds, you’re going to have way too many tabs and some memory problems. My teammate Toby Tripp came up with this:

def teardown
applescript «-EOS if macos?
tell application “System Events”
tell process “Firefox”
set frontmost to true
click menu item “Close Tab” of menu “File” of menu bar 1
end tell
end tell
EOS
end

This code snippet goes inside

class FirefoxBrowser < Browser

inside the plugins/javascript_test/lib/javascript_test.rb file. We could have just killed Firefox but we needed to play nice in case any other builds were using Firefox at that moment (we test with Selenium to use Firefox to click through the app).

Now all you have to do is change the position of the browser.teardown call in that same file in the ‘define’ method. By default, teardown is called just before the end of the block created by

@browsers.each do

browser

But I want it to get called after every test so you can put it just before the end of the block created by:

@tests.each do

test

So that the end result looks like this:

def define
trap(“INT”) { @server.shutdown }
t = Thread.new { @server.start }

# run all combinations of browsers and tests  
@browsers.each do |browser|  
  if browser.supported?  
    browser.setup  
    @tests.each do |test|  
      browser.visit(" http://localhost:4711#{test}?resultsURL=http://localhost:4711/results&t=" + ("%.6f" % Time.now.to_f))  
      result = @queue.pop  
      puts "#{test} on #{browser}: #{result}"  
      @result = false unless result == 'SUCCESS'  
      browser.teardown  
    end  
  else  
    puts "Skipping #{browser}, not supported on this OS"  
  end  
  # browser.teardown  
end  
 @server.shutdown  
 t.join      end  

You can see by the comment where the teardown used to be. So now when you run your test the tabs will close when finished. In my next post I’ll talk about how to fail the build when the javascript tests fail.

Oh, and I’d like to thank Mike Ward for turning me on to javascript unit testing in the first place.

Generating a Unique Number

Thu, 17 Jan 2008 04:35:00 +0000

Weird requirement at work today: We needed a number that was alpha-numeric, exactly 11 digits, unique, and non-sequential. At first we thought of using a hash, but MD5 and Sha1 give you way too many digits. We could truncate to 11, but not knowing much about hashing, that made me pretty nervous that we’d have collisions. After a bunch of discussion, we decided on this:

MAXIMUM_FOR_10_DIGITS = 0xffffffffff
(MAXIMUM_FOR_10_DIGITS + cart_id + Time.now.to_i + rand(100_000_000)).to_s(16)

I was thinking about using hex, but I was worried about what would happen if the number got too big and went to 12 digits. So I did this calculation:

irb(main):001:0> 0xfffffffffff - 0x10000000000
=> 16492674416639

Turns out there’s all sorts of room between the lowest and highest 11 digit hex number. Paul and Schubert did some quick calculations assuming lots of carts per day and figured out that we have about 250,000 years before we run out of space. Now since we use a random number, there is a chance that we could get two matching numbers so we do a database lookup to make sure that a generated number hasn’t been used before. There is still a small chance that two identical numbers could be generated in the time between generation and saving to the database, but I think we can live with that.

Update: After some interesting comments, I wrote a follow up post. Check it out for more math goodness.

The Method Hit List

Thu, 10 Jan 2008 02:01:00 +0000

In a previous post I talked about how my team has a daily metrics build that reports on our code quality. One of the things we measure is the Flog score of our methods (Flog is a ruby program that evaluates the complexity of your methods). We have a standing developer task to look at the top five worst methods in our application and reduce their complexity. Every iteration we try to spend some time looking at a few developer tasks, which are things that don’t provide any direct business value but do make our code better, automate some manual process, or fix some technical problem not tied to a bug.

Yesterday Chirdeep and I refactored the worst method in our app and while it wasn’t easy it produced a number of benefits:

First, about halfway though the refactoring we came upon some twisted logic that seemed as if it could never evaluate to false. After pulling in Ali for some consultation, we realized that indeed it would always be true and therefore was a bug. We fixed it and moved on.

Second, we noticed that the object inherited from another object, but overwrote the constructor with a strange one that took different arguments. This had the effect of making other objects interact weirdly with this class in order to deal with its ‘sorta’ inheritance. We managed to get rid of the custom constructor and have it rely on it’s super class.

Finally, we noticed a nice clear division of labor in the complex method so we were able to extract a method that had a single responsibility. So we ended up with two methods that shared the complexity of the original one.

I would have been happy if all we accomplished was the last item. However, when you take a look at your most complex methods you often find bad stuff hiding amongst the craziness.

It’s also important to note what we didn’t do:
We did not simply extract out every 5 lines into a method. While that would have improved our Flog score per method, it would been the code equivalent of sweeping dirt under the rug. Methods should, as much as possible, have a single responsibility. If they don’t then all you’re doing is moving a bunch of complex stuff into many different arbitrary places, instead of just one place. Which is actually worse. So, like all metrics, complexity is something to pay attention to but you don’t want to slavishly chase after good numbers.

Svn Merge from Trunk to Branch

Tue, 08 Jan 2008 03:21:00 +0000

Every time I need to do this I try to use svn’s –help which is only possible to decipher if you already know what to do. Then I spend too much time on the internet looking for the answer. And finally I break down and ask my tech lead. With much shame. I’m putting this up here so I least I’ll know where to find it. Maybe you too.

In the directory of the branch:

svn merge -r 1001:1002 https://svn.dev.your/repo/trunk/src .

If you checked in some files in revision 1002, then what you’re saying is that you only want the changes from that checkin with ‘-r 1001:1002’ Which is followed by the url of the trunk (where the changes were checked in) and a ‘.’ to say that you want to merge to the current directory. Use ‘–dry-run’ if you want to see what would happen without actually screwing anything up. After the merge is successful, you now have some modified files in the branch you can check in.

Done and done.

Mocking Backticks and Other Kernel Methods

Thu, 15 Nov 2007 03:20:00 +0000

There’s a bunch of places in our build where we need to execute a system command (such as running sqlplus) but often times we’ve found that the command fails and our build happily churns away. It’s pretty easy to check the result of system command with $?.success? but we have to remember to do that everywhere… So that means it’s time to extract a method. Here’s what Kurtis and I came up with:

  module DatabaseHelper
   def self.command_runner(command)
     output = `#{command}`
     puts output
     fail unless $?.success?
     fail if output.include? "ERROR"
     output
   end
  end

So you pass in a command as a string and the command runner:

runs the command
prints out the output
fails if the return code is bad
fails if the output includes “ERROR” (useful when running database imports with sqlplus)
and returns the output just to be a good citizen

So that’s cool but when I started to write the module I thought “Hey, I can test this.” Which is a cool side effect of extracting methods from your rake file. So cool that you probably want to consider it even if the method won’t be re-used.

Anyway, I started to write the test and realized that I would need to mock the backtick, which is a Kernel method. I’m using mocha so I tried:

Kernel.expects(:`)

and

Kernel.any_instance.expects(:`)

But no luck. Turns out you need to mock the class in which the backtick method will be called. So here’s the tests I ended up with:

unit_tests do

 test "command runner executes a command" do
   DatabaseHelper.expects(:`).returns("some output")
   $?.expects(:success?).returns(true)

   output = DatabaseHelper.command_runner("command")
   assert_equal "some output", output
 end

 test "command runner fails if ERROR in output" do
   DatabaseHelper.expects(:`).returns("ERROR: this is broken")
   $?.expects(:success?).returns(true)

   assert_raises(RuntimeError) do
     DatabaseHelper.command_runner("command")
   end
 end

 test "command runner fails if process fails" do
   DatabaseHelper.expects(:`)
   $?.expects(:success?).returns(false)

   assert_raises(RuntimeError) do
     DatabaseHelper.command_runner("command")
   end
 end
end

I was having such a good time mocking that I later went back and mocked out the puts so I didn’t have to see the output when running the tests. Gotta keep things clean.

Btw, you may notice that I’m not using Rspec a work anymore. Yep, it’s true. My new team is using DUST (well, the code that would later be part of the inspiration for DUST) which is pretty cool. I still prefer Rspec, but I can’t really make the case that we should switch to Rspec mid-stream (if we were using straight Rails testing, I would make that case.).

Employee Appreciation

Thu, 25 Oct 2007 01:21:00 +0000

As a consultant, I spend a lot of time at big companies and big companies generally have a morale problem. Bureaucracy, cubicles, and lax management will tend to do that. Typically they try to solve this problem by exchanging money for goodwill. A few projects ago I was consulting at one such company where they spent a ton of money having a day of games, burgers, and events during business hours. There was a big ad campaign, lots of promotional tie-ins and give-aways – a team of people clearly worked on this for weeks (or maybe months). Now add up productivity, food, and sumo-suit rental costs and we are talking some serious money. All worth it in service of employee good will, right?

A few days after this spectacle, my team and I decided to walk to lunch. Now the most direct route to our restaurant of choice happened to be through the front door. However, the security guy stopped us when we tried to leave. Employees aren’t allowed to enter or leave through the front door. Why? That’s the policy.

So we backtracked, went out another door, had lunch, and came back. At which point we discovered that our badges wouldn’t let us in the side doors. So we tried another side door. No luck. We went around to the front where we found out that security had turned off our badges intentionally so that we would have to go in the front door and get a lecture from the head security guy. I’m not kidding. 3 grown men got a talking to for trying to exit through the front door.

Later we heard, informally, that the company wants to keep the front entrance looking nice so that’s why employees may not enter or leave through the front door. The obvious implication here is that visitors would be put off by the sight of employees walking in through the same door as them. Does anyone really believe that one day of corporate sponsored fun is likely to make up for the daily humiliation of having to enter through the back door of your own company?

Metrics for Rails

Fri, 14 Sep 2007 04:31:00 +0000

Everyone thinks they write good code – it’s just part of human nature. You can’t do something every day and not secretly suspect that you’re good at it. Self-delusion is a powerful thing so you need to use metrics to take a hard look at your code.

On my current project, we’ve just added a daily metrics build (run every day at midnight by CruiseControl.rb) that takes a look at our code in three ways:

Code coverage with Rcov
Cyclomatic complexity with Saikuro
And um…, Flogging with Flog

Rcov is a code coverage tool that can be used with Rails Rcov to add a bunch of rake tasks to your build so you can figure out which lines of code are run by your tests… and which are not.

Saikuro computes cyclomatic complexity which “measures the number of linearly independent paths through a program’s source code.” Methods with more paths are harder to understand/debug/modify.

And Flog is cyclomatic complexity with an attitude. It scores ruby methods with an “ABC metric: Assignments, Branches, Calls, with particular attention placed on calls.”

Why do we use both Saikuro and Flog? Well Flog catches Ruby specific complexities that cyclomatic complexity doesn’t (for instance, calls to eval are given particular weight) and it picks up methods that Saikuro misses (we use metaprogramming to define a fair amount of methods and Saikuro seems to miss anything not defined with a ‘def’). But Flog outputs a flog score which isn’t very familiar to most developers, while cyclomatic complexity is a relatively well understood computer science term.

Also flog has the scariest looking software website ever. I can’t believe those guys have the chutzpa to put such pictures on a site that advertises them as consultants for hire.

So how did our code look? Pretty good – 96% code coverage overall but some methods need testing love. And some of our code had pretty high cyclomatic complexity or Flog numbers so we’ll need to write some developer tasks to fix the problems uncovered. (Every iteration we work on a few developer tasks in addition to the development, bug, and production tasks.)

But the metrics build isn’t done, in the future I’d like to:

Find duplicate code with CPD.
Use Heckle to see if our tests are any good. (Heckle changes your code (not permanently) and then re-runs your tests – if they don’t fail then you’ve got bad tests)
Fail the build on bad metrics numbers.
Figure out how to integrate the results Panopticode-style with all the cool visualizations it offers.

Why 50% Test Coverage Seems More Painfull Than No Test Coverage

Fri, 31 Aug 2007 03:02:00 +0000

Recently I was on a project where a bunch of code had been written before we arrived. It was quite a struggle to get the application under test. After a number of months the team hit 50% and then we just stayed there. We had a hard time getting client developer buy-in on the push upward from 50%. I didn’t really understand this attitude at first, but after talking with the devs, I realized that the tests were mostly a nuisance for them. They saw it like this:

“If I have to gut a few pages, as part of a changing requirement, now I also have to spend a day fixing the stupid tests. And the tests never really catch any bugs, so what was the point? All the tests are doing is slowing me down.”

Since the coverage was low and many of the test writers were new to unit tests we didn’t really have a lot of protection from bugs. But we also had a sizable suite to maintain. They were feeling all the pain of keeping a test suite running but seeing none of the benefits. Now, as a guy who’d been on projects with good test coverage I could see how the suite was making things better: First, I had a roughly 50% chance of being able to use the tests as documentation for any class I opened up. And second, the tests were catching some problems before they made it into the repository – but not enough for the client devs to notice.

We never did convince them that writing tests allowed them to go faster in the long run and it caused me to wonder how many teams get to 30-60% coverage and give up because the benefits are too subtle and the pain is too much?

Renting American Cars

Thu, 30 Aug 2007 01:59:00 +0000

As a traveling consultant I’ve developed a pretty poor opinion of American cars. Sliding doors that won’t close, unfathomable glove boxes, ugly/boring interiors, boring/ugly exteriors, crazy placements for the radio/ac/heating/trunk release buttons, plastic doors with tails of extra material hanging off poorly drilled holes… The other day I was sitting in a Chrysler 300 talking with one of our tech leads about how he really wishes the rental company hadn’t upgraded him and would’ve much preferred a Hyundai Elantra – a car that costs half as much as the 300. It’s a bit depressing.

Writing Your Own Custom Log Parser

Wed, 29 Aug 2007 04:01:00 +0000

I spent the last week writing a log parser of all things. The web site I’m currently working on is the sign-up page for a large company so they are very concerned about how far the average joe gets into the process before they give up. If everybody bails on the address page, then perhaps it’s too complicated or unresponsive or… something. Up till now we’ve been redesigning pages based on some data and a lot of guesswork so we thought it might be about time to get some serious data on every session every day. Now the first thing you learn about parsing rails logs is that they are designed to be human readable but not so much computer parsable. An Apache request is one line. A Rails request log has lots more info, but is multi-line. And since we run a lot of Mongrel instances at once, all these requests are interleaved, so sorting them out is an intense computation (we actually ran out of memory a few times before we did some optimization).

Our first pass was mostly a ruby script that called off to a few classes which produced a csv file that our analysts could play with. But they wanted more data – data only Apache could provide. So we added an Apache log parser and merged the results. And there were bugs (stuff wasn’t logged, sessions seemed to end at the starting page or somehow start in the middle (both of which are impossible)). And the code was complicated and hard to understand. At this point we realized that we had been treating this task like a second class citizen – a task to be automated but not really cared about. So we bit the bullet and refactored the log parser to be more Object Oriented. Now there’s a log collector that has lots of requests. Each request mostly delegates to its Apache Request or its Rails request. Along the way things became more clear and the bugs started to become obvious, including one I have to share with you:

Remember how I said many sessions seemed to start in the middle while others ended at the beginning? Well, it turns out that a typical telemarketer will sign up one user and then head back to the first page to sign up another. At which point Rails grabs the session id of their previous session and logs it before processing the request (which wipes the session, but too late for the logging). So it looks like the previous session ended at the beginning. Of course, the next session has had it’s beginning stolen so it looks like it started in the middle. Since Rails does this bit of logging before it even looks at our code we needed to adjust our logger to ignore the false session and look for the real session id in a extra bit of logging we added in.

Anyway now that we have our own custom log parser we can get some awesome info on our app for debugging weird stuff that only happens under load and the analysts get a csv file then can import into Excel and generate all sorts of cool graphs and statistics. Yes it did mean a week of writing regular expressions and some crazy command line statements like:

cat access.log | grep | awk | sort | uniq -c | sort

but we now have a very sharp, well tested, tool.

RSpec on Rails: Models

Fri, 10 Aug 2007 04:58:00 +0000

In my last post I talked about using RSpec without Rails, but since just about all my Ruby programming involves Rails, I should probably get into how to specify Rails code.

RSpec on Rails: Models

So if you’re new to RSpec and you want to get started quickly, head on over to rspec.rubyforge.org, get RSpec, Spec::Rails and then, after you’ve created a new Rails project and rspec-ed it (all detailed on the website) you can run:

ruby script/generate rspec_scaffold person name:string
        phone_number:integer
        cash:decimal

Which will get you a bunch of stuff to play with. Of you’ll need to set up your database.yml and run rake db:migrate if you want thing to work.

Fire up your favorite IDE and find the person_spec.rb and you should see something like:

require File.dirname(__FILE__) + '/../spec_helper'

describe Person do
 before(:each) do
   @person = Person.new
 end

 it "should be valid" do
   @person.should be_valid
 end
end

Not the world’s most rigorous set of specs, but then our model doesn’t have much behavior at the moment. You’ll notice that the auto-generated spec does use RSpec’s built in interrogative feature. ‘be_valid’ makes RSpec look for ‘valid?’ Running the spec with spec formatting (‘-fs’) produces the following output:

Person
- should be valid

Finished in 0.028878 seconds

1 example, 0 failures

But what’s interesting is that if I remove the string between it and do, the output remains the same. If you leave off the name of a spec, RSpec guesses a name for you by looking at what you’re asserting.

But let’s say I want to specify that a Person is invalid without a name. Here’s how I would do that:

 it "should be invalid without a name" do
   @person.should have(1).error_on(:name)
   @person.should_not be_valid
 end

As you can see, there’s some built in sugar to make checking errors more readable. Now that I have a failing spec I can add the following line to my Person model to make the it pass:

 validates_presence_of :name

Now let’s do something a little more interesting:

 it "should run inside a transaction when making rich" do
   Person.should_receive(:transaction).and_yield
   @person.make_rich
   @person.cash.should == 1000000.00
 end

What I’m saying here is that a Person should have a method that makes them rich and furthermore that the rich-making should happen inside a transaction. RSpec has mature mocking/stubbing framework built into it. ‘and_yield’ can even specify what to yield so you could return an object that you created in the spec and then check to make sure some things happened to it. Of course, before I get to much further, I should write some code to make the spec pass:

 def make_rich
   Person.transaction do
     self.cash = 1000000.00
   end
 end

Now what if you already have a codebase that has a lot of Test::Unit tests but you want to start using rspec? Well you could just start writing specs in their own spec folder and rspec’s default rails task will run both your tests and your specs.

Next time we’ll look at view specing with RSpec.

Future note: No we won’t and don’t do that - 2018/09/10

RSpec without Rails

Wed, 08 Aug 2007 02:53:00 +0000

I’m going to try and turn my recent presentation on RSpec into a series of blog posts – This being the first. Before I get on with it I would like to thank the Houston Ruby and Rails Users Group for having me. They asked some pretty insightful questions and were generally a great audience. I’d also like to thank ThoughtWorks for paying for my flight, rental car, and lost billable hours so I could do this presentation.

RSpec without Rails

So in order to show you how RSpec works, I’m going to start with specifying a class that deals with string case like so:

require File.dirname(__FILE__) + '/../spec_helper'
require 'string_caser'

describe StringCaser do
  it "should upcase a string" do
    caser = StringCaser.new("A String")
    caser.upcase.should == "A STRING"
  end
end

After requiring a spec_helper and the file I intend to spec, I set up a ‘describe’ block. Inside the block there is a single spec (or example) which is also a block. (If you’ve looked at RSpec before you might wonder what happened to context and specify -- they’ve been replaced by ‘describe’ and ‘it’ In the example I declare a the name of the spec to be “should upcase a string” which does not execute and could be any name you like. By convention they all start with “should.” Then I create a new StringCaser and give it a string. On the next line I call caser.upcase which should equal “A STRING”. Instead of having to remember which is expected and which is actual, the RSpec way is to simply do what you want and then call ‘should’ on the output. Much more readable. If you’re wondering, should and should_not can be called on any Ruby Object.

So If I run the above spec without creating the file, I’ll get an error on the require. So let’s create a file called string_caser_spec.rb

class StringCaser
  def initialize(content)
    @content = content
  end

  def upcase
    @content
  end
end

And if run the spec like so:

spec string_caser_spec

I get:

F

1)
'StringCaser should upcase a string' FAILED
expected "A STRING", got "A String" (using ==)
./spec/models/string\_caser\_spec.rb:7:

Finished in 0.029125 seconds

1 example, 1 failure

Because I forgot to upcase the return value. Oh well, easily fixed:

def upcase
  @content.upcase
end

Now run rspec so that it formats the output like so:

spec string\_caser\_spec -fs

And I get this output:

StringCaser
- should upcase a string

Finished in 0.036034 seconds

1 example, 0 failures

Lets add another right below the previous spec:

it "should know if a sting is all lowercase" do
  caser = StringCaser.new("a lowercase string")
  caser.should be\_lower\_case
end

Now I’m saying that a StringCaser instance should know if its content is all lowercase. But this time I’m using RSpec’s built in support of interrogatives. When RSpec sees ‘be_’ in a matcher (which is the thing that comes after should), it looks for a method with a name that follows ‘be_’ and has a ‘?’ at the end. In this case ‘be_lower_case’ makes RSpec look for a method called ‘lower_case?’ and calls it. There a lot of nice syntactic sugar like this in RSpec – way more than I could ever hope to cover. Now ‘lower_case?’ doesn’t exist yet so I get an error, and then I write this code:

def lower_case?
  @content.downcase == @content
end

And I’m back to passing. Btw, you can call rspec with -c to print your output in color, if you like.

Now for something a little more interesting. I want StringCaser to throw a RuntimeError if it receives an object that does not respond to ‘upcase.’ Here’s the spec:

[1, 3.4, Object.new ].each do |bad_thing|
  it "should raise a Runtime Error when passed a #{bad_thing.class} because it doesn't respond to 'upcase'" do
    lambda { StringCaser.new(bad_thing) }.should raise_error(RuntimeError)
  end
end

Cool, huh? Sure I could do this in Test::Unit, but it would have to be one test with a generic name. In RSpec I can name on the fly (yes I could use some Ruby magic to define methods at runtime in Test::Unit, but think of the readability! (or lack there of)).

So now I add some code to StringCaser:

def initialize(content)
  raise RuntimeError unless content.respond_to?(:upcase)
  @content = content
end

and when I run my specs with spec string_caser_spec -fs, I get:

StringCaser
- should upcase a string
- should know if a sting is all lowercase
- should raise a Runtime Error when passed a Fixnum because it doesn't respond to 'upcase'
- should raise a Runtime Error when passed a Float because it doesn't respond to 'upcase'
- should raise a Runtime Error when passed a Object because it doesn't respond to 'upcase'

Finished in 0.035137 seconds

5 examples, 0 failures

Which is nice. There’s also a way to get this output as an html file.

Now I find that as I’m writing unit tests that I often end up with a bunch of different setups, or one big complicated setup that only 1/5 of which is important to any one test. Either case kinda sucks, but RSpec gives me a way to segregate my setups like so:

describe "StringCaser with all lower case contents" do
before(:each) do
  @caser = StringCaser.new("lowercase")
end

it "should return true when lower_case? is called on it" do
  @caser.should be\_lower\_case
end

# More specs for this case
end

describe "StringCaser with some upper case contents" do
before(:each) do
  @caser = StringCaser.new("MiXcAsE")
end

it "should return false when lower_case? is called on it" do
  @caser.should\_not be\_lower_case
end

# More specs for this case
end

Now I can I have a place where I specify for each context. This tends to drive out more specs and produces a more complete suite. Warning, this does not mean I’m endorsing huge setup methods. A spec should be readable in and of itself. And yet, I think a short setup that does exactly what the ‘describe’ says is a useful thing.

Next time I’ll talk about testing/specing/examplifying Rails Models.

RSpec Resources

Mon, 06 Aug 2007 15:25:00 +0000

Later this evening I’m giving a talk on RSpec at the “Houston Ruby and Rails Group.”

In case somebody misses a link, I’m posting them here:

RSpec’s homepage: rspec.rubyforge.org

RSpec Users Mailing list: rubyforge.org/mailman/listinfo/rspec-users

David Chelimsky’s Blog: blog.davidchelimsky.net

If you install RSpec as a plugin to your Rails project, then look inside: vendor/plugin/rspec/examples for some nice spec examples

Once you’ve got RSpec and RSpec on Rails installed in your project, running:

  ruby script/generate rspec_scaffold person name:string phone_number:integer cash:decimal

from the base of your project will get you a bunch of generated specs to look at and play with.

I'm Presenting on RSpec in Houston

Sat, 28 Jul 2007 14:09:00 +0000

I’ve been invited to give a presentation on RSpec at the Houston Area Ruby and Rails User Group on Monday August 6th – and I’m pretty excited to do it. It’ll be a nice opportunity to organize my thoughts and lessons learned after using RSpec on my last two projects. I’ll be talking about the why and the how of RSpec (focusing mostly on the how), specifically as it applies to testing Rails apps.

Watch this space for the blogization of this presentation.

Caching Dynamically Modified Objects and the Trouble it Causes

Tue, 03 Jul 2007 05:57:00 +0000

So I told you, in a previous post, about how our application has two parts: the live site and the cms. And that we want objects of the same class to behave differently in the two applications (only ‘live’ status objects should show up on the live site), which is not so hard in Ruby. But a few days later and after some conversations with our architect about how our caching works we realized that we had a huge bug.

We use memcached and cache-fu to cache our objects and we share that cache between the cms and the live site for the simple reason that we want an update to the object in cms to blow away the cache for the live site (otherwise the live site would never see the change as it would constantly be referencing the old cached version). But, as you may have already guessed, if the live site gets an object cached from the cms then all sorts of ‘draft’ stuff will show up on the public site because an object built in the cms will not have the extra special dynamically added add ons. Which is no good in a very serious way.

So after a long bit of trying to dynamically tap dance our way out of this mess, we realized (and by ‘we’ I mean Mike Ward) that the solution was to separate the caches with different namespaces and make sure the save callback blows away both. So all the objects cached by the cms have a prefix of ‘cms’ and all the live site objects have a prefix of ‘live’ and they never touch each other’s objects. Good. Then we added in an after_save hook that destroys both ‘cms-1234’ and ‘live-1234’ making sure that updated items will be seen as updated in both sites.

Yep, its a slight (probably very slight) performance hit now that the cms and the live site don’t share the same cache, but now we’re free to make the two rails instances behave the way they should without fear that they will get all messed up with each other. Nobody likes objects that sleep around.

Big Weekend for Nerds

Sun, 01 Jul 2007 23:18:00 +0000

Lots of chatter around ThoughtWorks about the iPhone but, for me, this weekend was all about the Weird Al concert. I got your alpha geek right here, buddy:

Great performance – More costume changes than a Vegas show. And more Star Wars parodies too.

Why does modifying my classes dynamically only work once in Rails?

Wed, 27 Jun 2007 02:31:00 +0000

(Or the downside of class caching)

This is a gotcha in Rails where your custom plugin or other dynamic change of a class only seems to work once.

First, a little background:
Recently we had a problem where we wanted an object to behave differently in two different instances of Rails. Ya see, we have models that are shared by two separately running rails instances. One is a cms (content management system) where all objects (let’s say they are of class Thing) should show up. The other is the live site that should only show ‘things’ that have been designated ‘live’ (as opposed to, say, ‘draft’). Now changing the search to filter out non-live ‘things’ was easy, but since you can call thing_instance.related_things and get back related objects of type Thing, now we have a problem. In the live site I want a call to ‘related_things’ to only bring back ‘live’ objects. But in the cms I want the same call to ‘related_things’ to bring back all the related objects no matter what the status.

Now I could create a new method called ‘live_related_things’ which wraps ‘related_things’ and filters but there’s 3 problems with this:

‘related_things’ is used EVERYWHERE in the project – I’d have to change, like, a million files.
Having ‘related_things’ and ‘live_related_things’ is really dumb. It’s a bug waiting to happen.
Hey this is ruby – I can change the behavior of an class dynamically. Wooo!

So I created a plugin that uses alias_method_chain to change the behavior of the live site. And I felt all cool ‘cause I was using THE POWER OF DYNAMIC LANGUAGES to solve a problem. And when I tried it in the app, it worked great. Once. On the second request it didn’t work at all.

One of the cool things about Rails is that you can change your models and you don’t have to restart your server – really speeds up development. But how do they accomplish such magic? By tearing apart and rebuilding your models on every request. And the plugins only get loaded once. So when I started up my server the plugin loaded and everything was fine and on the second request the model was rebuilt from scratch (and the plugin was not reloaded) and it didn’t work at all. Now this wouldn’t happen if I was running in production mode (because its crazy inefficient to keep reloading classes on every request). So my solution was to add this line to my application.rb:

extend ThingExtension if Dependencies.mechanism == :load

‘Dependencies.mechanism == :load’ is true if rails is loading classes for every request and ThingExtension is a module that looks like this:

module ThingExtension
unless Thing.include?(::ThingLiveFilter)
load File.dirname(__FILE__) + ‘/../../local/plugins/live_filter/init.rb’
end
end

I have to call ‘load’ instead of ‘require’ because require only loads the file once and it’s already been required when the project started up. ‘Load’ will open that file every time, but I need to check if Thing already has the ThingLiveFilter module included in it or I’ll get one of those great ‘stack level too deep’ errors. Why? Well the when the project starts up, the plugin is called and then if ‘load’ calls it again alias_method_chain will get stuck in an infinite loop trying to redefine a method that’s already been chained.

So there ya go, something to keep any eye out for when you’re developing in Rails.

Update: if you’re gonna do something like I described above AND cache your objects then you might want to read my bit on Caching Dynamically Modified Objects and the Trouble it Causes

Fixture Migrations in Rails

Fri, 15 Jun 2007 19:41:00 +0000

(This is the third of three posts on Rails fixtures)

Awhile back Josh Cronemeyer approached me with an idea to migrate fixtures in Rails. The problem we were trying to solve is when your database changes you need to edit all the fixture files to reflect the change – which usually doesn’t happen and then things break/get weird. We spent a few hours hashing out the basic plan (load all the fixtures into the db, migrate, then write the db out to the fixture files) and writing some code. Then he went off, tested it, and made it work with (very) occasional input from me. He’s written up a nice post on the topic called How to Migrate Fixtures in Rails and has posted the code at Google Code.

Go check it out if you hate migrating fixtures by hand.

What are Rails Fixtures Good for Anyway?

Fri, 15 Jun 2007 19:30:00 +0000

(This is the second of three posts on Rails fixtures)

I like using fixtures for setting up a development environment. Applications need data in the db to function so what most teams do is pass around a database, or two, or five. A new person joins the project and he/she gets a dump from one of the devs, imports it and goes to work. What happens when you screw up your db by:

Running a bad migration
Importing a crap load of data for a task (I remember once filling up a db with all movies playing in North America) but later you don’t need it.
Doing something dumb. (my personal favorite)

Well, you need to get somebody else’s db. Which maybe in a bad state and it definitely doesn’t have the same data you’re used to because they’ve been changing it a lot.

But if you have a bunch of fixtures checked in to source control, you can always go back to a base state. Anytime something is acting wacky and you think it might be a data problem (which happens more and more as the app gets older) wiping the db is simple and easy. Also, everyone has the same data so you can collaborate easier.

Next: How to migrate fixtures.

Why I Hate Using Rails Fixtures in Tests/Specs

Fri, 15 Jun 2007 19:25:00 +0000

(This is the first of three posts on Rails fixtures)

So you have a Person object that has a many to many relationship with Roles and Movies (maybe this is site where people track their favorite movies). So to test it you need to have the following files:

people.yml
roles.yml
movies.yml
and the join tables:
people_roles.yml
movies_people.yml

If you want to test that only people of a certain role can edit another person’s favorite movie list, then your test is spread out amongst 7 files (don’t forget the test file and the actual class file). Maddening enough when you’re writing it, but super crazy insane to debug when the test breaks. More than likely a developer will delete/comment out the the test when it breaks, or they will change it so passes but breaks functionality.

Long tests that repeat themselves look bad, from a DRY (don’t repeat yourself) point of view but consider this: A fixture that is useful for 10 tests tells you nothing about any one test. Many times I’ve faced a broken test that relies on a bloated fixture and been pretty well stumped. There’s all sorts of crap on this thing that have nothing to do with the test I’m trying to fix, so now I need to understand 10 tests to fix one. That takes time and makes people hate testing.

See my next post for why fixtures are still useful.

See also Jay Fields’s Testing: Inline Setup and Josh Cronemeyer’s Are Rails Test Fixtures Good or Evil?

Public Static Final for Ruby

Wed, 30 May 2007 02:26:00 +0000

When I first got into Ruby awhile back I found that I wanted to do something like I used to in Java:

public static final String BLUE = “blue”;

But darned if I could get the internet to tell me how, so here I am giving back to the community:

in solar_constants.rb:

module SolarConstants
MASS_OF_EARTH = 5.98e24
end

in some other file:

require ‘solar_constants’
puts SolarConstants::MASS_OF_EARTH

or you can:

require ‘solar_constants’
class Gravity
include SolarConstants

def print_mass_of_earth
puts MASS_OF_EARTH
end
end

This being Ruby you can, of course, redefine a constant. But you will get a stern warning.

EVDO Card on a Dell D620 Running Linux

Tue, 29 May 2007 14:03:00 +0000

One of the things that kept me from installing Linux on my laptop was concern about support for peripherals such as printers, digital cameras, USB devices in general, and my ThoughtWorks-provided EVDO card. The cool thing with Fiesty Fawn is that all except the EVDO card worked out of the box (I should say I had some trouble with my wireless on Edgy Eft). And the EVDO card only took about 5 minutes to get working after Peter Ryan pointed me to this article:

Get broadband wireless with Verizon EVDO and Linux

It’s a few command line scripts to run and currently I haven’t figured out how to turn it off or configure image detail (like you can with Verizon’s software), but it works.

The Last Day of Rails Conf 2007

Tue, 22 May 2007 05:49:00 +0000

A light day for me as a 4 hour flight + 2 hour time change + wanting to see my wife for a few hours this weekend means I left early.

Before I get started I wanted to say a few words about Portland:

It’s a great city. The train took us from the airport straight to the convention center (and our hotel which was 2 blocks away). Then we found out the train is free in the downtown area. So we had super easy access to bars/food/good times. Also the city, while being big enough to support an ecosystem of coolness is small enough to avoid being overwhelming.

Keynote – The Rails Way with Jamis Buck and Michael Koziarski:

General format of this talk was here’s a submission we got for inclusion into Rails core and here is what’s wrong with it. Oddly enough they called the people out by name – the ones who had the bad code. Not sure how I’d feel about being outed like that. [Correction: Koz informs me that people had submitted their code to The Rails Way (not core) and were asking to have their code ‘publicly refactored’ – I totally missed that] Mostly I thought their advice about how to DRY up you code, but not at the expense of readability, was pretty good. I was distracted by the fact that the guy sitting in front of me had obviously slept on a chocolate bar. Did he not know that pieces of chocolate were sticking to his back? Or not care? Perhaps I should have told him.

JRuby on Rails with Charles Nutter and Thomas Enebo:

Well this was the topic of the moment and the presentation was pretty packed even though it was the last day of the conference and they had given a longer version of this talk as a tutorial. Their answer for why would you want to run Ruby on the JVM?

Full access to Java’s Unicode support – which is way better than Ruby’s
System threads instead of Ruby’s green threads
All of Java’s libraries are available
A Rails War file can be dropped into any app server – so its way more deploy friendly for Java shops
It’s faster than Ruby’s implementation

Not a bad set of reasons. What’s the downside? Well not all gems work – if they have C in ‘em they need to be ported. So no RMagick for you.

They ended the session by creating an app from scratch, turning it into a War file, and deploying it using Glassfish (an open source app server from Sun) using a JDBC driver.

Impressive.

All in all Rails Conf was good stuff. I want to thank the presenters and organizers for all their hard work.

Oh hey, if you run into Josh Cronemeyer get him to tell you about his late night experiences with the cross-dressing, freakazoid, marching band at a bar called Dante’s late Saturday night. Its a good story. Better yet, get him to blog about it. I think he has pictures.

Thoughts on the Second Day of Rails Conf 2007

Sun, 20 May 2007 15:28:00 +0000

Keynote (AKA paid sponsor time) – Cyndi Mitchell (ThoughtWorks) and Tim Bray (Sun):

Cyndi seemed a bit nervous and well she should be – this is ThoughtWorks big play to mold the face of the Ruby/Rails world by announcing RubyWorks. RubyWorks is a “free, open source LAMP production stack is for RedHat Enterprise Edition and CentOS, and will allow users to quickly put Rails applications on production servers.” Pull it down, drop in yer Rails app, and you gots a server. It’s all free, but if maybe you’d like some support we could perhaps sell you some. And we’ll support JRuby. Pretty big move for us as a company. So why would we expose ourselves to this risk? Other than the tons of money we stand to make, ThoughtWorks feels like the Java/.NET world is all wrapped up with horrible bloatware and bad management practices and while we’re still interested in fighting for change in those areas we feel think there’s an opportunity to make a real impact in a wide open area. And it won’t be open for long. There’s a a lot of huge companies interested in Rails but worried it’s not enterprise ready. How long will it take before there’s a Websphere for Rails? And a Wasad/Rad for Rails? How long before programming in Ruby becomes just like programming in Java because the company you work for signed some huge deal with “Big IT” and you have to use the approved R2EE stack?

Best to get out in front of that and make a non-evil way to reassure large companies that using Rails is “safe”. Exciting and funny presentation – but it’s possible I’m biased.

Tim Bray got up and talked some more about JRuby and at one point he called Cyndi back up to talk about why ThoughtWorks is deploying Mingle in JRuby and she basically said it was a way to get big companies to feel better about deploying it. But was really cool was to see ThoughtWorks and Sun freakin’ Microsystems sharing the same stage and talking as equals. My little company (little compared to Sun) has grown up a bit. But I think it’s really cool that we’re still willing to take risks and make big moves. Our support of Ruby a year ago looked risky, and I’m hoping we’ll look back on ThoughtWorks support of JRuby and RubyWorks a year from now and feel the same way we do now about getting into the Ruby/Rails space.

Custom Rails Helpers – Glenn Vanderburg:

Good stuff. This is a prime example of one of those talks where I totally new a bunch of what he was saying already, but I picked up a few things (like using content_tag instead of just dumping out the string of an html tag).

Glenn name checked Erlang which made him the second presenter of the day to do so. Showing that 12 minute video at the last Rails conf sure did resonate with some people. I vote that at next year comparisons to Erlang be made mandatory for all sessions.

The Business of Rails – Panel:

This was pure Rails pron and I enjoyed the heck out of it. 5 dudes who stuck out on their own and started up a Rails consulting company sat around and told war stories that were completely entertaining. Joe O’Brien recounted how he signed a fixed bid project that included “basic email functionality” that ended up becoming a re-write of Gmail for 20K. Everyone told how they fell on love with ruby and the cockles of our hearts where warmed.

The Dark Art of Developing Plugins – James Adam:

Very funny and informative. James started out with super Gothic lettering on the screen and a total Satan voice as he inducted us into “THE DARK ARTS OF DEVELOPING RAILS PLUGINS” which was hilarious. The good news is there’s not much to it. You put the things you want to do in an init.rb file and the files init.rb needs go in a lib folder and any rake tasks go into a tasks folder and the rest is just dealing with how Ruby extends/includes things. If you want to add instance methods to classes you use include, but you need to use extend for class methods. You can overwrite included on an object to make it include and extend what you want. At this moment I was going: “Oh, that’s why every plugin I’ve opened up has separate InstanceMethods and ClassMethods modules inside another module, and hey yeah they all do have an included method… Neat” I had been wondering that for awhile.

Open Mic Demo Session:

This is hopefully going to become a staple at Rails Conf (they did it last year too) as it’s all sorts of wild. 10 presenters get 10 minutes to show something they’ve worked on. It ranged from the incomprehensible where some dude was talking about a yaml configuration plugin thing and half the dudes around me were muttering “what does this thing do?.” To the wacky, awesome, and probably useless Statisfy.net where you can put a bit of javascript on yer webpage and see real time hits to your site on a Google map (“hey someone in Africa just looked at my site!”) To the useful and cool: ThoughtWorks very own Alex Verkhovsky showed CruiseControl.rb. In 3 minutes he unziped the file, aimed cruise at an svn repo, and started up the server. Bang zoom there was a continuous integration site up and running. Pretty cool. Another cool demo was health portal where you could cut and paste any article from the site into your own custom page. It was all drag and drop and you could place the items exactly where you wanted them on the screen with a very cool online wysiwyg editor. I wish I remember the name of the site. Rats. [It’s revolutionhealth.com – thanks commenters]

I know a lot of ThoughtWorkers who worked crazy long hours to make the ThoughtWorks hangout/demo/vendor area so very cool and it was totally worth it – thank you so much. The joint was truly buzzing with activity. I remember one pretty typical moment where the following things were all happening:

I was working with Josh Cronemeyer on our fixture thing.
A Mingle demo was in full swing.
A small crowd was watching our looping video about ThoughtWorks
People were pairing on the Guv’nor app (using CruiseControl.rb and Mingle of course)
Dudes where chatting about JRuby (including some of the core team) in the lounge (sipping fancy fancy carbonated grapefruit juice)
Martin Fowler was holding court with some gushing fans.

Well done.

First Day of Rails Conf 2007

Sat, 19 May 2007 04:33:00 +0000

Finished the first day of Rails Conf and here are my impressions:

DHH’s keynote:
Lets celebrate what we have. Blah blah blah. Rails has come a long way. Yadda yadda yadda. Here’s a sneak preview of what ActiveResource is going to be like in Rails 2.0. Ooooh, Aaaah.

Actually the ActiveResource stuff was pretty cool. With a little bit of setup you can have one controller method serve up xml and text and html and all sorts of stuff. All RESTful style of course. And the debugger will be back (broken when someone fixed a bug in Ruby 1.85) and it will be a real debugger.

When will 2.0 be out? He didn’t say. Go use edge if you dare.

Bob Martin’s Clean Code Presentation:
On the one hand, this is a lot of stuff I’ve heard before (I hung around Object Mentor at the start of my career), but Uncle Bob is such a good speaker he got me all fired up again about TDD, Red Green Refactor, and incremental steps. There was this point where a slide was clearly missing from the presentation that showed what code he would write next, and he totally recovered so fast as to make you think he planned it. Then some guy in the audience tried to call him out on it and Bob turned his mistake into a virtue and expounded on the idea. Mad presentation skills: he’s got ‘em.

He used RSpec in his tests and that was awesome (the guy next to me actually cheered when he saw the tests were in RSpec). I’d like to take a moment here and yell at the Rails Conf organizers for having no RSpec talks, ‘cause RSpec is easily one of the cooler things happening in Ruby/Rails. Why the freeze out? The jerk store called and its running out of Rails Conf organizers.

Oh and I liked that there was a lot of code in Bob Martin’s talk. No pretty pictures and hand waving here.

Scale Rails Without Bounds on Amazon EC2:

Speaking of pretty pictures and hand waving…

I mean the idea of renting server time for 10 cents an hour is pretty cool. And being able to automagically kick off a bunch of servers when you need ‘em (in 2-8 minutes) is freakin’ bad ass. But where’s the rubber meets the road details? Can we see some code? Nope. Boxes with arrows pointing to cylinders is all you get.

Btw, you better back up your data all the time ‘cause when those servers go down you are totally out ‘o luck (perhaps you might like to use Amazon’s S3 (super soldier storage) for that?).

Standing on the shoulders Giants
Was too damn crowded so I couldn’t get in. So Josh Cronemeyer and I paired on out plan to migrate fixtures – coming soon.

Spam I Have Known by Jim Weirich:

Awesome – Good speaker and a fascinating topic.

As Jim has been managing the Ruby Garden Wiki for the last few years, he has run into his fair share of spam (in this case, spam that has links to web other sites to give them the “Google Juice”). Some of the spam was stupid and hilarious (“I’m sorry for links my kids and my wife need food”). But some was super clever – Cross site scripting , zero height links, and pseudo encrypted text that used JavaScript to decrypt itself into more JavaScript.

Anyway, his solution was to create a “tar pit” for spammers. When they come in and are identified as spammers (through various combinations of content, IP address, lack of reverse DNS look up, being anonymous, etc.) they take the post and send it to a separate db and make it look like the spammer has succeeded. This means that if they come back to check to see if their spam is still there – they see a site with spam. Everyone else sees a site that is spam free. They do this because the spammers tended to learn from rejection. Rejected?, try again. And again. So every solution was overcome. By denning them feedback (and making it appear they have succeeded) Jim W. hopes to hold them off for longer. He incorporated this idea and many other spam prevention measures into a wiki called “Ruse.”

Later I fell in to conversation with Roy Singham, Martin Fowler, Ola Bini, DHH, and bunch of other luminary types. Then a limo pulled up and whisked them away. Guess I don’t quite rate a limo yet. Sigh.

Buffalo burgers and beer at Voleur would have to suffice. Nice place, even if they did screw up our order. Twice. But that meant that we got to meet the chef who had a naked lady tattooed on his arm. Very cool.

Oh and I should mention that the ThoughtWorks hang out area is very cool. We’re building a real live app using Mingle and the very Mac mini’s I worked on at First Street Live. You too could get the chance to pair with a real live ThoughtWorker (gasp). Also there are Mingle (a very cool agile project management tool written by us) demos and lounge chairs and free soda pop and cute little give aways. Mad props to all who set this up.

Beware Tomcat5.5 in Feisty Fawn

Tue, 01 May 2007 03:11:00 +0000

I moved to the newest version of Ubuntu over the weekend: Feisty Fawn. I was having troubles getting my wireless working (Intel PRO/Wireless 3945) and I had heard it works in Feisty. The upgrade, um, took out my system. Yep. Everything was chugging along just fine until it tried to upgrade Tomcat and then I got a cryptic error message saying the update had failed and I should email them. No real indication of what to do next. So I tried to get the upgrade running again, but I’m not savvy enough with the Linux. Tried to restart the machine… Big mistake. Not so much with the booting. So I had to wipe that partition and reinstall. I didn’t lose anything except time.

Then today when I was setting up my machine I couldn’t get Tomcat5.5 to work (we use SOLR which uses Tomcat behind the scenes for fast searching). Lost more time trying to figure out what was wrong. Finally I thought maybe Aptitude’s version was bad, so I removed it and got Tomcat straight from Apache. Everything works fine now. So if you’re moving from Edgy to Feisty, maybe should remove Tomcat from the package management before upgrading.

Good news is that my wireless works now. Long way to go, however.

Secret Origin of the ‘Super Ultra Critical Showstopper’ Bug

Tue, 24 Apr 2007 01:17:00 +0000

When you first start a project and you set up bug tracking software there tends to be 3 levels of bugs: Low, Medium, and High. After awhile you might notice there are now ‘critical’ bugs. Come back a little while later and there’ll be a ‘showstopper’ or ‘blocker’ bug level. And the funny thing is, most of the bugs will be logged at the highest levels. Literally 50% of bugs are given such a status as to make you think the app is in shambles. Except that when you actually open up qny of them to see what could possibly be so horrible, you find things like: missing fields, pages that should be pop-ups, and misspelled text. How is it that these things are given the highest level of importance?

Today, while I was working on a super ultra critical showstopper bug, I had some time to reflect on this phenomena. The bug was this: One of our models was missing a field. And the business wanted this field to be entered before the user could perform some task. Now, I should point out that nothing in the app was giving an error message, except that the user could get away without entering a field that had no functionality except to be displayed on the screen later. In my humble opinion, the app could have gone live without fixing this bug and nothing terrible would have happened.

But, you might say, it’s not my call to decide that. Absolutely right. But it’s also true that sitting next to it, at the same level of priority, were bugs that really were super ultra critical (such as the temporary inability to edit anything at all in our content management system – yipes!). So how do we let the business decide what gets done first but also have a rational system of ranking bugs? Well you need to have ranking system that actually means something. If high isn’t defined then everything gets rated as ‘high’ (especially cause manager X is gonna freak if it doesn’t get fixed – and every bug has a manager X) so we create ‘critical’ and then everyone calls things critical, and this continues until you have a very long list of bug ratings (and only the top two will ever get used). On a previous project we had the same problem and so moved to this rating system:

Priority 1: The app is down. No part of the app runs at all.
Priority 2: A large part of the app is not functional, but some parts still work.
Priority 3: An important function does not work, but there is a workaround.
Priority 4: Satellite functionality does not work.
Priority 5: Cosmetic change.

This worked pretty well: important things got fixed and if the client wanted to get something moved up in priority it could always be played as a card (like while technically misspelling the name of the application is a cosmetic change and therefore priority 5, but still…)

Chicago in Mid April

Thu, 12 Apr 2007 03:49:00 +0000

Snow turned to freezing rain which turned to rain and then, just to mess with me and my 30 minute walk, tiny hail. The hail hurt my face. That’ll teach me to brag about my commute.

Stubbing/Mocking a Partial Within a Partial with RSpec

Tue, 10 Apr 2007 05:07:00 +0000

So in a previous post I complained about not being able to mock/stub a partial from within a partial, but with Peter Ryan and Mike Ward’s help we got it all figured out.
Normally in your test(spec) you call a partial like this:

render :partial => ‘partial_name’,
locals => {:page_variable => mock_page_variable }

But if you want to intercept all or some calls to render with partial, them you can’t do that. But what you can do is call a partial like so:

render ‘_partial_name’

If you need some locals you can stub them out like so:

@controller.template.stub!(:page_variable).
and_return(mock_page_variable)
render ‘_partial_name’

So if I wanted to stub out all calls to render, and assert that the string “Blargh” appears on the page, it would look a little something like this:

@controller.template.stub!(:render)
@controller.template.stub!(:page_variable).
and_return(mock_page_variable)
render ‘_partial_name’

response.body.should include(“Blargh”)

Now why does that work while “render :partial =>…” does not? I, uh, really don’t know. Ideas?

My Commute

Sat, 31 Mar 2007 16:24:00 +0000

Working in downtown Chicago has its advantages – everyday includes a scenic walk along the Chicago River:

And after a hard day of delivering business value, who doesn’t enjoy a movie and a nice drink?

Windows vs OSX vs Ubuntu for Ruby/Rails Development

Thu, 29 Mar 2007 03:43:00 +0000

When I first started working with Rails, I had a windows laptop and so that’s where I worked. Later I got on two different Rails projects that used OSX (on Mac minis). Now I’m using Kubuntu. I thought I’d take a minute to compare them all.

I’m not going to mince words here – try to avoid developing in Windows if at all possible for these reasons:

Tests will run very slow (by a factor of 10 or more).
The mysql drivers are not so good (When I developed in windows, I noticed I kept losing my connection to the database during operations that took more than a second or two and I couldn’t get any of the replacement drivers to work).
A fair amount of the plugins will be much harder to install (we are using the Raspell gem for “did you mean” functionality which needs aspell and the windows dudes still haven’t figured out how to install it. On Ubuntu, it was a simple “aptitude install”).
Help online is usually not geared towards the Rails on Windows developer.
If you have to develop in Windows, get IntelliJ and load up Cygwin

If money is no object go for a Mac:

TextMate is about as good as it gets for Rails Development (It has a bunch of readily available macros and pretty darn good auto completion).
All the cool kids use Macs for development so there’s plenty of help online.
They’re very pretty and fun to use (this sounds like a lame reason but if you’re going to be using something for 8 hours a day it might as well bring you some joy).
However, sometimes the package management on the Mac let me down. Fink and Ports and all that tend to fight it out and getting image magic installed ended up taking down the whole system (we had to wipe it – no foolin’).

If you think Mac’s are overpriced, then go with Ubuntu (or some other linux distro).

It’s free!
It has a rockin’ package management system.
IntelliJ (with the Ruby plugin) is pretty darn good and I have a feeling its going to get better.

Really the choice between OSX and Linux is mostly the choice between TextMate and IntelliJ (or Eclipse/RadRails). And, for now, I think TextMate has a slight edge. TextMate’s auto complete is a joy to work with while IntelliJ’s indexing makes for crazy fast searching. In fact once IntelliJ gets a bit better (don’t let me down Jet Brains – help a brother out with some autocomplete (and maybe some refactoring support)) then it’s just a choice between a really slick operating system and merely a functional one.

Using OpenStruct to Enhance Your Mocks

Sat, 24 Mar 2007 20:36:00 +0000

So I use RSpec’s built in mocking framework for my mocking/stubbing and it works quite well. I sent up a mock like so:

mock_active_record_instance =
mock(“give it a name here, but probably a better one than this”)

and put some expectations on it like so:

mock_active_record_instance.should_receive(:id).and_return(1)

but some calls to the mock I don’t care about so I can stub them out like so:

mock_active_record_instance.stub!(:name).
and_return(“Arthur, King of the Britons”)

If I don’t need it to return anything I can leave off the and_return and it will return nil. But it’s tedious to do that. Lately I tend to use OpenStructs as my mocks like so:

mock_active_record_instance = OpenStruct.new
mock_active_record_instance.should_receive(:id).and_return(1)

Now here’s what’s cool about that: My spec(test) will fail if the expectation of a call to id isn’t met, but any other calls will be ignored (and nil returned). If I had made mock_active_record_instance a mock, like in the first example, I’d have to mock or stub out all the calls it’s going to get and that is just painful. This way I can ignore the stuff I don’t want to spec(test) painlessly.

You can also do fun stuff like so:

mock_active_record_instance = OpenStruct.new(:height => “5’10’’”,
:weight => 260,
:cars => [“hyundai”, “saturn”])

So now mock_active_record_instance.cars will return an array with 2 stings. And height will return “5’10’’” and weight will return 260. Maybe, in the future, a call to weight will return a smaller number – I’m working on it.

Warning – it may be that you have to do this because too many things are going on in the method you are specifying(testing). A better solution might be to figure out how to shorten up that method.

Jay Fields wrote about using OpenStructs in a series of blog posts about stubbing in rails which are totally worth reading.

Rails partials should be explicitly called with locals

Tue, 20 Mar 2007 00:47:00 +0000

Since Rails partials can see all the @variables of their parent view it’s pretty common to see partials with @’s all over the place. But the cool thing about partials is reusing them in other views and if you do you have to remember to populate all the stuff it needs and name everything the way it wants. Bah, I say. More and more I find myself explicitly calling partials with locals even if seems a bit redundant:

<%= render(:partial => ‘content/poll’,
:locals => {:poll => @poll}) %>

It goes along with this “partials are like methods” analogy I’ve been thinking about lately. And in big complicated pages with lots ‘o partials everything seems more clear. What do you think?

Mocking/Stubbing partials and helper methods in RSpec view specs

Fri, 16 Mar 2007 07:05:00 +0000

In my previous post I whined about not knowing how to mock/stub partials and helper methods in RSpec view specs. Now I have the answer and I’d like to thank the RSpec users mailing list and David Chelimsky for taking my phone call.

In short, it’s all about @controller.template

@controller.template.stub!(:a_helper_method).and_return(true)

Stubs out the appropriately named a_helper_method and returns true. You can mock it out also like so:

@controller.template.should_receive(:a_helper_method).
at_least(:once).and_return(true)

Which is like the stub except that it checks to see that a_helper_method was called at least once

For partials you do this:

@controller.template.should_receive(:render).
with(:partial => ‘form’, :locals => {:cookie => mock_cookie})

Which checks to make sure the partial _form.rhtml is rendered and that the view attempted to pass mock_cookie in as a local. Since I didn’t specify a return it will return nil and I don’t have to worry about what’s in the _form view (of course, I’ll spec it by itself). Cool.

There’s just one problem: You can’t mock/stub a partial inside a partial. Let’s say that inside _form is another partial and while I was specifying _form I wanted to mock it out like so:

@controller.template.should_receive(:render).
with(:partial => ‘piece_of_form’)

But later in the spec when I do this:

render :partial => ‘form’

I’m making a call to the template to render and it gets intercepted by the mocking framework! So not only do I get an error that the wrong thing was rendered (_form instead of _piece_of_form) but I’ve intercepted the call so I don’t get inside the view at all. Even if I were to expect the call to render _form I make inside the spec (which is weird) I wouldn’t be able to render the _form. Bummer.

Oh well, I got almost all of what I wanted.

What I want from view specs

Fri, 16 Mar 2007 00:33:00 +0000

I got an email today from a friend asking me about specifying views in RSpec in which he admitted he hadn’t done it much and first reaction was “I know what you mean!” Because it’s hard to spec views when you’re not hitting the database. You have to mock or stub out so many calls to your object just to get at the one or two lines you want to spec. And then I realized why I hate specifying views so much: It’s exactly like specifying a huge method. There’s all this stuff going on and you only care about one little bit and so you end up with 10 lines of mocking/stubbing to get to 3 lines of what you really want to say. Now with a ginormous method I have the option of pulling out little pieces of functionality into smaller methods and going straight at them. Then I can stub out those smaller methods when I’m specifying the big one. I can do this a little by creating a bunch of partials and specifying them individually, but when I want to spec the parent of all those partials I need to be able to stub them out. But I can’t figure out how to do that. The same goes for helper methods. I spec those individually already so I don’t want to spec them again but I can’t figure out how to stub them out. If I could do these two things I think specifying views would be much less painful.

Off to the internet for answers!

Wrapping a Context in a Context (in RSpec)

Wed, 14 Mar 2007 02:31:00 +0000

A few days ago I blogged about solving a sticky problem by overwriting the “new” method on an ActiveRecord object inside a test (spec). Something I should have mentioned is that this is dangerous because if another spec (test) tries to call new and gets executed after the dangerous spec/test is run, then of course it won’t have the original new method. And it’s a very hard error to track down unless you know it’s been redefined. The solution to this, should you need to do it (and you probably won’t – any good mocking framework will probably do what you want), is to wrap a context around all the contexts that need it and use context_setup like this:

context ‘When the situation demands it’ do
context_setup do
Class SomeObject
def some_method_to_overwrite
#do whatever I want – Bwaa ha ha!
end
end
end

context ‘Some context’ do
#specs go here
end

context ‘Some other context’ do
#more specs go here
end
end

After it drops out of the context, the method isn’t redefined anymore. Yeah! Of course, to be real good and safe you should probably alias the new method in the setup and reassign it back in the teardown. But this works too.

Rcov/RSpec problem solved in RSpec 0.8.2

Tue, 13 Mar 2007 01:24:00 +0000

I loves me some RSpec, but I’ve had this consistent problem on my last two projects when RSpec and Rcov are run together: Sometimes you get Seg Faults. In a previous post I talked about my solution to this (redefining the new method on ActiveRecord objects), but it was a hacky solution and I’m glad to say that it’s out of the code base as of today. On a hunch this weekend I installed RSpec 0.8.2 (we had been using 0.7.5.1), took out my super-fun hacktacular solution from a spec and it ran fine.

But why?

Looking through RSpec’s list of bugs fixed I didn’t see any reference to the bug I logged a few months ago (I didn’t really expect too – they were never able to reproduce it) but I do have any idea. David Chelimsky (lead dev of RSpec) had been telling me for awhile about how he was going to change RSpec to stop using method_missing on object because when Rails takes it for its own purposes, then RSpec has to some hacky things to steal it back (David wrote about it in “rspec.should use_a_little_less_magic“and so did I in “Battle of the method_missings”). Since the removal of that code was the big change between 0.7.5.1 and 0.8.2, I’m gonna guess that it was the problem.

Lemme solve all your problems

Thu, 08 Mar 2007 02:29:00 +0000

Persistent, baffling problem?

Windows: restart computer
Linux: change file permissions

I need to Tattoo this on the back of my hand so I can see it while I type.

Long day.

Battle of the method_missings

Wed, 07 Mar 2007 00:41:00 +0000

So if you’ve heard about Ruby, you’ve probably heard about method_missing (short summary: if an object receives a message it doesn’t understand it calls method_missing which you can overwrite. This allows Rails ActiveRecord to handle User.find_by_email_and_name_and_phone_number on the fly – it parses the message and gives you the method you want dynamically). But when too many things try to use it at once you get some interesting problems. At some point RSpec enabled calling object_instance.should_include(‘thing’) by using method_missing on Object to catch any message that started with should_. So, in the previous example, RSpec would look for a method called include? on the object_instance, pass ‘thing’ into that, and tell you about it if it failed. Which is pretty cool because instead of saying:

assert_equal [‘a’, ‘b’, ‘c’], object_instance.some_method

you could say:

object_instance.should_eql [‘a’, ‘b’, ‘c’]

which is more in line with ruby’s quest for code that looks like natural language.

But, when you use RSpec with Rails the method_missings fight. RSpec overrides method_missing and then Rails steals it back and you get all sorts of bugs. Sure you can do some kinda ugly stuff to steal it back again, but then the Rails source changes and you have another bug to chase. So with the release of Rspec 0.8 there is a new syntax supported and the old way has been deprecated (the maintainers of Rspec promise that they will provide a conversion script before they take the old syntax out of the framework). So

object_instance.should_include(‘thing’)

becomes:

object_instance.should include(‘thing’)

Why? Because now RSpec just has to add a ‘should’ method to the Object class which takes in an operator. No method missing on Object at all, so Rails can do all the fanciness it wants.

There are two implications here. One, if you’re writing a ruby framework you should be careful about method_missing as your users may want it too. And two, if you’re thinking about adding method_missing to an object that inherits from a Rails object, beware.

FoxyProxy is bad news

Thu, 01 Mar 2007 04:52:00 +0000

At work we have to go through a proxy to get to the outside world. So every time I take my laptop home I need to switch the proxy setting on my browser. Lame. Surely there is some sort of Firefox plugin to help me with this tedious task.

And there is. But Foxyproxy isn’t it.

It’s crazy heavyweight and unpredictable. The heavyweight part is that you can define different proxy settings and each of those settings can have white and black lists (with wildcard or regex matching) that decide whether or not to use a proxy. Or you can mix and match proxies for different sites. All that I could live with, but I just spent a few hours tracking down a bug that didn’t exist because of Foxyproxy’s weirdness.

The feature I was working on involved calling out to an outside system. But the browser just spun and spun when I tried it. After a LOT of time spent talking to people about the right configuration for this 3rd party system I guessed that just possibly this was a proxy issue and uninstalled Froxyproxy (and used Firefox’s built in proxy system). And it worked like a charm.

I went back and looked at my settings for Foxyproxy and I couldn’t for the life of me see how it wasn’t using the proxy to connect to the third party system. And yet it was ignoring the proxy for this one call and one call only. Very hard to debug, as it’s one of the last things you’d expect to stop working.

My coworkers are saying good things about SwitchProxy, but I’m a little bit skittish after getting burned on this.

ActionMailer tips

Tue, 27 Feb 2007 01:16:00 +0000

My latest story card is all about getting our app to send email so I’ve had to delve into ActionMailer. It’s a cool way to send emails without tons of code but there are some things that are a little bit wacky. Let us say that you’ve got a cookie website and you want people to be able to send emails of their favorite recipes to each other. “My that’s a handsome cookie” they might say and “Oh look, an email button that lets me send this recipe to a friend. This site has met my needs exactly.” In order to get such a thing to happen you would create a RecipeMailer model that extended ActionMailer kinda like this:

class RecipeMailer < ActionMailer::Base

def recipe_email(email_params)
@from = email_params[:from]
@recipients = email_params[:recipients]
@subject = email_params[:subject]
@sent_on = email_params[:sent_on]

@body[:recipe] = email_params[:recipe]
end
end

In your controller you’d call:

RecipeMailer.deliver_recipe_email(email_params)

And some good old method_missing magic would match deliver_recipe_email with your recipe_email method which would then render a view.

Er, a view?

That’s right, the email templates are rhtml views. You write them as you would a view, but ActionMailer turns them into emails. So one might look like this:

Howdy neighbor,
Here’s a recipe you might like:
<%= @recipe.text %>

And Rails would of course evaluate the <%= stuff like it would in a view. But, although it’s a view, it doesn’t have access to any helper methods you have defined, even if you put them in ApplicationHelper (which are supposed to be available to all views). You’ll have to declare this in your RecipeMailer model if you want the view to see them.

ApplicationHelper:
helper :application

Another thing to watch out for is any links you include in your email must have the full path to your app. When using link_to you can pass in these options:

:only_path => false, :host => @host

Where host is the host you get off of request.host but you won’t have access to request in your view or your mailer, so you’ll have to pass it in from your controller (again, the email template is only sort of a view). But if you’ve added some fancy custom tags (like a hypothetical recipe_image_tag that takes in a recipe object and does all sorts of neato stuff) you might find that the combination of needing a full path and being rendered by an email client (ie not your rails app) will force you to, sigh, hack up your own links like so:

Not the prettiest code I’ve ever written but sometimes time is a factor.

The stack trace that doesn’t mean anything

Fri, 23 Feb 2007 05:03:00 +0000

We’re using an open source java app to help with searching on my current project and when I start it up (through cygwin – must… stop… developing… in… windows…) it throws up a bunch of stack traces. The first time I saw this I thought “holy crap, I must have configured it wrong.” And I was right. But then I configured it correctly and I was still getting a few stack traces on startup. I spent an hour or so trying to fix the problem before I remembered that lots of times a Java app will thow some errors at ya just for fun and I started up the rails app, navigated to the search, and it worked fine. Nice. So I just learned to ignore those stack traces.

Can you see where this is going?

Yep, later the search stopped working in a spectacular way that took down the whole app. Except I wasn’t sure it was the search breaking it. And I lost a bunch of time because I was ignoring the errors that were telling me something was wrong. They were different errors than the usual ones, but I was so used to hitting enter and watching a big stream of stuff fly by that I didn’t look at it.

Errors that are not critical errors need to be stuffed in a log file. Otherwise you’re training your users to ignore your output. Take a look at how Mongrel starts up:

** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with development environment…
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready. INT => stop (no restart).
** Mongrel available at 0.0.0.0:3000
** Use CTRL-C to stop.

A few lines and no random crap. Want to know the details? There’s a log file for that.

Neal Ford has been having similar problems with garrulous software.

Rails fixtures on Linux and Windows

Thu, 22 Feb 2007 00:14:00 +0000

At my current gig I develop on Windows while the rest of the team is working on Linux (but not for long, I just got PartitionMagic and I’m going to dual-boot). This has caused several problems and one of the most vexing is that the Linux dudes can check in fixtures that blow up on my Windows box. Every time it happens I take a look at the offending fixture and it isn’t putting something into a column that has been declared ‘not null.’ So I fix it by adding in some data, but why the hell isn’t it blowing up on their machines (or cruise control, which is on a Linux box)? My colleague Ricky Lui figured it out today. Allow me to quote from his email:

On our Linux machines, MySQL is installed with no default sql_mode. That means
if there’s a certain column that’s supposedly NOT NULL, your unit tests can
still pass even without setting that column (e.g. through test fixtures).

For MySQL installation on Windows, the default sql_mode is set, so that same
unit test that passes on a linux machine will fail on Windows.

I think it’s a good idea to let MySQL catch more errors for you. You can
view and set your sql_mode:

mysql> SELECT @@global.sql_mode;

mysql> SET @@global.sql_mode = traditional;

As per http://dev.mysql.com/doc/refman/5.0/en/server-sql-mode.html, the
‘traditional’ mode is ‘ “give an error instead of a warning” when inserting
an incorrect value into a column.’

Nice job Ricky.

You want fries with that?

Wed, 21 Feb 2007 00:11:00 +0000

I’ve got some bad new for ya. I know you’ve been reading this blog picturing me as some handsome muscled young stud – but I am not. I’m fat and 35 and all muscle has left since I stopped working out about 5 years ago. What happened? Who cares – what I wanna talk about today is how every waiter is trying to kill me.

They are!

As a traveling consultant I tend to eat out 3 meals a day while I’m on the road. And in order to lose some weight I thought I’d just stop getting side dishes. Hamburger – no fries. Pasta – no garlic bread. Whatever it comes with – I don’t want it. This is very hard for waitstaff. Here’s a conversation I’ve had about a thousand times:

Me - “I’ll have the French dip, but I don’t want any sides”

Waiter - (confused pause)

Waiter – “You don’t want… Would you like to substitute something else for the fries?”

Me – “Nope. Just the French dip is fine.”

Waiter – (weirded-out pause) “Um… Okay.”

15 minutes later

Waiter – “Here ya go.”

Me – “I thought I ordered no sides, but there are fries covering half the plate.”

Waiter – “I guess the cook just put them on anyway. You don’t have to eat ‘um”

Now I gotta decide if I feel like sharing the completely shameful fact that if those fries are left in front of me I will eat every single one of them along with about a gallon of ketchup. But I usually just shut up and eat the damn fries.

The perils of keeping your prototype

Tue, 20 Feb 2007 01:32:00 +0000

Oh sure everybody says you should throw out the prototype after you make it, but look at all that yummy code just waiting to be used. Sure you wrote it fast and without tests – but it took a lot of work and it would be silly not to just use it as the base of your real project. Brush it off, clean it up a little, and it’ll be fine.

Don’t do it.

First of all it’s not fine. Code written without the threat of production tends not to be good. But let’s pretend you did write some nice code (you didn’t but this is pretend time). Now you’ve got huge chunks of untested code floating around that looks like it does stuff but actually is just left over from the prototype. Joe dev comes along and tries to write some functionality and he comes across a bunch of prototype code. It’s really confusing. It’s doing all this stuff. Stuff that doesn’t seem to work like he thought the app was supposed to work… Huh. “Oh well, I guess that’s how they want it done so I better follow along.” he thinks. Or maybe there’s a bug and he’s looking through code to see what’s going on. It’s not a lot of fun to realize you’ve spent a few hours rummaging around dead code. Or the case of the new functionality that seems to already be done. Just a little bit of hacking and it IS done. “Except for those extra methods that I couldn’t figure out… Better just leave it ‘cause they must be important.”

It may seem silly to keep the prototype separate from the real project, especially if you’re going to be referring to it all the time. I don’t care if you do end up copying and pasting much of the prototype into the real code as you add functionality. At least this way every line you add is a conscious decision. There’s less chance of your code becoming a way confusing combination of used and unused, polished and unpolished, tested and legacy.

Theater and Programming

Fri, 16 Feb 2007 00:41:00 +0000

Awhile back I was bored of the suburbs and being a teacher (high school physics for 7 years – no foolin’) so I decided to become an actor. I didn’t quit my job, but I did use my summer vacation to take 3 acting classes at once. It was a crazy good time and I completely recommend it to anyone looking for a change of pace and/or the opportunity to meet lots of pretty girls. Anyway, one of my classes was Improvisation which should really be called game playing for adults. Remember cops and robbers? Well yeah, it was like that. Teacher would get some volunteers and come up with some wacky situation and the 2 or more actors would just play out the scene. You’re a salesman on his way to a big sales call who sits next to a weird guy on the bus. And you, you’re a crazy person who thinks this bus is a submarine. Go.

Somewhat scary, but lots of fun. But like cops and robbers the game could stall if people didn’t play fair. “I shot you. No you didn’t” and so on. The number one rule for all improv is that you always say yes. If the butcher sez that its 5pm and closing time, you aren’t allowed to say “No, its 3 o’clock.” Everything just stops, the trust is lost, and the bit just becomes a boring argument. There’s something magic about saying yes. Which is why there’s a rule in XP that when someone asks for help, you always have to say yes. Even if you don’t want to. Even if you don’t know how to help. It just hurts the team too much when everyone is free to say no.

Now, of course, this rule is flexible. There are plenty of cases you can think up where it would be perfectly reasonable to say “No.” But it’s a good guideline. I’ve seen teams that are great and teams that suck and the ones where you can always get someone to help you fall into the first camp. Saying yes maybe scary (like improve) because you might look like a fool (also like improve) if you don’t know what you’re doing. But when everyone starts doing it the fear disappears and the team really gels.

Object::InstanceMethods is a confusing error

Thu, 15 Feb 2007 00:10:00 +0000

So the project I’m working on has multiple databases – which is, in Rails opinion, kinda weird. Lately we’ve been having some issues where one developer will check in a migration that creates a new database and updates the database.example file. Everyone else does an update, tries to run their migrations, and gets an error saying that you can’t overwrite Object::InstanceMethods. What they need to do is add a few more lines to their database.yml file from the example file, but how would they know that unless they saw the database.example file amongst the stream of files that came down when they updated from source control. We need to get better about telling everyone (an email or a shout out to the room) when these get checked in. Because invariably some dev spends a few minutes scratching their head with this perplexing error.

What’s worse is that when I get this error on windows – It’s all over. I update my database file: same error. I drop and re-create all my databases manually: same error. The only thing I’ve found that works is if I kill my whole damn development folder and check out fresh. I realize that this makes no sense at all, but it works. Is windows hiding or caching some file somewhere? No idea. Luckily I’ve been mostly been developing on Linux for the last few days.

This error is another example of something I’ve been struggling with since I started writing Ruby: The errors tend to be unhelpful. Java and other strongly typed languages fail fast. If you give some method the wrong class, it blows up right away. In dynamic langs like Ruby the method doesn’t care what it gets so long as it can respond to the methods it called on it (the famous “duck typing”). Maybe the method just passes it along, or maybe it gets something off that the bad object and tries to do some stuff with it. So you can get pretty far down the stack from where you actually made your error before the interpreter blows up – Which makes debugging a challenge. Now I know this added flexibility is what makes Ruby so cool. But it’s not without its costs.

RSpec and Rcov are not playing nice

Wed, 14 Feb 2007 03:35:00 +0000

So I ran into this weird problem with RSpec (the unit test (spec) framework for Ruby) and Rcov (a code coverage tool also for Ruby). At some point, usually as your list of specs gets longer, if you run your specs with Rcov they blow up with a seg fault. I’ve seen this happen on OSX, Windows, and now Linux. On my last project we just had to give up running Rcov (which sucked). This time Ricky Lui and Peter Ryan managed to figure out that if they comment out the tests that have an Active Record object mocked out for new, then Rcov is fine. Say you have an object called User and your controller gets a new instance and does a bunch of stuff to it. Well, you want to test (specify) what happens to that object so you do this:

specify “something important should happen to User” do
mock_user = mock(“mock_user”)
User.should_receive(:new).and_return(mock_user)
mock_user.should_receive(:some_method).twice.and_return(“whatever”)
etc.
end

But Rcov no like this. So to get around this bug I had to open up my object in the spec and overwrite new, like so:

class User
def self.new
@@mock
end

def self.set_mock mock
@@mock = mock
end
end

So now I can make a call to User.set_mock and give it the mock I want new to return. Not quite as elegant as the mocking framework (and it doesn’t check to make sure new is called), but it works. But here’s the weird part: I can do some mocking of new in the spec, just not too much. In the spec that was making rcov bomb, I had 7 places where it read User.should_receive(:new). If I commented out 2, the spec ran fine with rcov. If I left all 7 in, it blew up. Weird, huh?

I need to get a small version of this consistently failing so I can give it to David Chelimsky, my friend and an a RSpec admin. I want to submit the bug to Rcov, but I can’t seem to find where to submit it. I guess I’ll just email whoever’s maintaining it.

Where’s the code?

Tue, 13 Feb 2007 01:34:00 +0000

So I wrote this website for my dad over the weekend. Mostly to replace this site I wrote about 8-9 years ago when I first learned HTML. Dad’s a sculptor and I thought it would be cool for him to have a web presence. Of course to update the site he has to edit these html templates I gave him, create thumbnails and non-huge regular images of the pictures he’s taken, and then ftp the files to his server. Not so fun for a guy born during the year of Pearl Harbor. So I wanted to create a site where he could easily create piece objects (with all the info), put them in galleries, and re-arrange the front page all through a web interface. Now a little over a year ago I did a similar thing for a friend of mine who sells homemade jewelry as my first Rails project and it took a while. Part of that was learning Ruby and Rails all at once, the other part was that I added some cool search functionality that she totally didn’t need, but mostly it was figuring out how to upload and manage images. That was a pain.

Now, armed with acts_as_attachment and a whole bunch more RoR experience, I knocked it out in a weekend. But the weirdest thing was that I didn’t write much code. Using the scaffold_resource generator I quickly threw up some pages to manage my 3 objects (FrontPage, Piece, and Gallery). Then I added some lines to Piece’s model to configure the attachments associated with it. I had to add a few more lines to make Piece’s views and controller deal with the attachments. And the rest of the time I’ve been screwing around with CSS trying to make it look nice.

The new site can not only upload images, but it automatically re-sizes them and creates thumbnails. Pretty bad-ass for a weekend. ‘Course I still have to deploy it.

What must not be spoken of

Sat, 10 Feb 2007 03:37:00 +0000

You join a new project (and if you’re a consultant like me then this happens a lot) and you start asking a bunch of questions. Where’s the svn repository? What version of Rails are we using? How do I install this other thing upon which our thing depends? And the answers come. But then it’s a few days later and the more sensitive questions come: Why aren’t we pairing much? Why is this meeting scheduled to take most of the day (when it should 45 minutes at most)? How come the dev boxes are so slow? And if the answers you get to those questions are variations on “Um… We can’t do that – talk to me about it later.” then something is busted.

You don’t know what it is, but you can feel it in the atmosphere. It’s tense. Some things are unspoken and mysterious forces are making sure it stays that way. Every iteration seems like it has too much in it. Its common knowledge that we have way more scheduled for release 1.0 then we can do in the time allowed.

If this is how XP feels then you’re doing it wrong. The point of an iteration isn’t to make every week a death march, it’s to increase visibility. If you didn’t get your 20 points of work done this week, the solution is not to mandate overtime or give speeches about how we “really need to get more done this week” and you certainly don’t want to just sign up for another 20 points for next week (only to fail again). If 2/5s of you team was out last week or lightning struck the servers then you can call it a bad week and move on. What bugs me is when people get pulled out for normal things (meetings, other projects they are assigned to, or normal fire dousing) and then the “bad iteration” gets blamed on “circumstance beyond our control” and we sign up for the same amount of work. There will, believe it or not, be more meetings in the future. So just call it overhead and adjust the amount of work we can accomplish in a week. And if that means we won’t hit our release date, then its good we know now. Right? If its not good we know now, then we’ve got a whole other set of problems.

Code Review

Fri, 09 Feb 2007 02:41:00 +0000

As part of the hiring process at ThoughtWorks we do a code review. After a candidate gets through a few gates (a resume’ screen and some phone interviews) the prospect is giving a choice of three code problems to work on. They have 3 days to complete the assignment and turn it in. Then we have 2 developers look over the code (separately) and assign it a score. A lot of these submissions are pretty bad. Java arrays used where an object would provide clarity, cryptic variable names, missing files needed to run the project, and (amazingly) code that just plain don’t work.

But last night I graded a submission that was a joy to read. He submitted in Ruby (a good sign, but not a sure thing by any means) and clearly understood the elegance of the language. He did some fun things with aliasing and Ruby’s built in collections methods that made me smile. And they weren’t just showing off either. Sometimes we get solutions that are crazy over designed. Pages and pages of code where 100 or so lines will do. But this was short and sweet.

I hope we hire him.

RMagick is a crazy pain

Tue, 06 Feb 2007 06:45:00 +0000

First of all I was sick for 4 days last week so, yes, I missed the first 4 days of my new project. Nice. Now this is an ongoing project (for about a monthish now), so it wasn’t a total tragedy. But I did feel like a mope calling in every day to say that I was indeed still sick. I’m still sick, but it’s at a point where I can manage it with OTC medications. Drugged up and billable I am. W00t. And did I mention it was -9 degrees outside with a wind chill of -20 F this morning? Good lord – that is a whole lot of cold (Especially as my new commute involves a brisk 30 minute walk across downtown Chicago). Good times.

But what I meant to talk about was RMagick – which is a pain. It’s an image manipulation gem for Ruby that is pretty powerful but totally took down 2 systems I tried to install it on in the last 2 weeks. Back on First Street Live, Clint and I tried to install it on a Mac mini but Mac Ports and Fink started to fight over who was the package management of record and ended up not even being able to find our gems. Why does every package management system assume it’s the only one you’ll ever need? They refuse to play nice with each other. We hosed that Mac mini up so bad we just gave up and wiped it. Damn we suck.

Then, because I refuse to be beaten by something (and I’m a glutton for punishment), I decided to put acts_as_attachment on a little side project website I’m building. Acts_as_attachment is a cool plugin for Rails which lets you link an image to an ActiveRecord class. It makes it real easy to upload images to your website and delete ‘em and resize ‘em and gussy ‘em up real nice. But it requires RMagick, which requires ImageMagick and, yes, there was another point where nothing worked on my machine. But, even though Windows takes a lot crap about being dev-hostile, I was able to recover and install. They key is following the straight and narrow as described at rmagick.rubyforge.org/install-faq.html. You could install ImagickMagick and Rmagick your self but that will never work (even though some of the documentation seems to imply that it would). Follow the advice of the page listed above and install the package ‘cause it will make your life easier.

If you’re looking to live on the edge you might want to check out attachment_fu which is a reworking of acts_as_attachment but with plans to make it more modular so you can replace the dreaded ImageMagick with image_science and file/database storage with Amazon S3 storage. It’s pretty beta right now, but there is some buzz about it.

Testing vs. Speed

Fri, 26 Jan 2007 05:09:00 +0000

So I tend to be on the more extreme side of the testing debate. On my current project I helped institute extensive use of RSpec and Selenium Remote Control. RSpec allows you to test the models, view, and controllers in isolation from each other through the use of a built in mocking framework (to mock out ActiveRecord objects and Stub out call to finders) and some ideas borrowed from Zen Test. Selenium Remote Control allows you to drive a browser (your choice) from Ruby (or most other langs). Which is cool because we were going to have lots of Javascript in this project to control the Google Maps features. The RSpec tests went well: Easy to write once you got a handle on their syntax, and they run real damn fast they don’t hit the database for the view and controller tests (their opinion is that model tests should still hit the db b/c models are so tied to the db it would be silly to test them without it). For the SeleniumRC tests we used RSpec as the harness and created page objects to query the page. Here’s a typical example:

context “Having not logged in” do

specify “going to the create_event page redirects to the login page
and upon logging in redirects back to the create_event page” do
navigate_to.home_page
home_page = get_home_page
home_page.go_to_create_event_page
login_page = get_login_page
login_page. login “Username” “Test”
get_create_event_page
end

end

The context specifies a given condition (having not logged in) but the text is not executed. The specify is a “specification” or what most people think of as a test. We have a navigator object that knows how to get around the app, a page_mother object that can get page objects, and page objects that know how to do and get things on a given page. So, to take the spec line by line:

navigate_to.home_page

Send the brower go to the right url.

home_page = get_home_page

Get the home_page object and run some assertions to check to see if the browser is really on the home_page.

home_page.go_to_create_event_page

The home_page object knows how to click a link that takes it to the create_event_page

login_page = get_login_page

The get_login_page method gets the login_page object and makes sure we are indeed on the login_page (we’ve been re-directed there as only logged in users can create events).

login_page.login “Username” “Test”

The login_page knows how to log in with a given username and password.

get_create_event_page

And finally the get_create_event_page method makes sure we are on the right page.

All tests are run in an instance_evel of a page_mother instance that has an instance of navigator returned by navigate_to.

Sound complex? Well it was. This was one of the most well tested apps I’ve ever worked on and, as it turned out, that was totally not what the client wanted.

Crap.

We spent a lot of time writing and running these tests because:

Google maps is hard to test – you have to write some fun javascript on your page to keep trac of any markers you place on the page because the GMap object won’t give them back to you once they’ve been set on it. And other stuff like that.

Google maps takes a bunch of time to render. We had to put our timeouts at 30 seconds and still tests failed b/c the map wasn’t back by the time we made an assertion. This, of course, caused the test to run way slow as lots of pages had GMaps.

The page_mother, navigator, and page object framework we invented was very elegant but pretty heavyweight. When you created a new page you had a lot of stuff to write just to test that page.

So our test suite, while impressive, ran slow and took a bunch of time to write. Which would have been fine if the client was interested in a bug-free site at all cost. Everybody on the ThoughtWorks team had just come from projects where we worked on apps that handled money and sensitive information. You really do not want to screw up that stuff. But his project was a little social network site where nobody would die if some data got lost. What we found out after 6 weeks is that the client just wanted something out there that looked good and in a hurry so they could see if anyone was interested in this site. Luckily when Obie and Zed joined the team they were able to see how much this reliability was slowing down the team and find out that the client didn’t much care about the value of rigorous testing. So we went in and ripped out all the SeleniumRC tests.

It was a sad day for me.

I was really proud of that test suite and it had taken a lot of hard work. Much more important was the fact that I had failed as a consultant. I had let an unspoken assumption slip through which stopped the team from delivering what the client wanted.

After we stopped writing and running Selenium tests (we still wrote unit tests with RSpec) we could go way faster and knock out a bunch of functionality in the last two weeks. Which is awesome: The site is looking good. Obie and Zed plan to go back and put in some Selenium tests once the site becomes more stable.

Now before I get beat-up for saying testing make you go slow, I should say this: I know that in the long term testing makes you faster. Speed you get now by not testing comes at the price of fragility and buggyness and trouble refactoring later. This is true. But for now the client is willing to accept a bit of ruff edges if it means speed – And we, as consultants, have to respect that.

Huda's Wedding

Wed, 24 Jan 2007 02:50:00 +0000

Getting married requires paper work

A lot of paperwork:

Huda, his wife Uzma, and Huda’s parents

Andy’s not so sure about the other Andy.

I kiss my wife

Their first dance as married people.

My wife and some drunk guy we met.

The ladies of the family

Take more photos!

In which I complain about air travel

Tue, 23 Jan 2007 05:17:00 +0000

So I work in Atlantic Beach, FL and I live in Chicago. So that means I spend a bunch of time on a plane every week. But this week I was best man in a wedding… in Seattle, WA. You know, the exact opposite corner of the lower 48 and not so easy to get to, as it turns out. I had cleverly arranged to connect through Chicago so I could meet my wife at O’Hare and then fly together to the land of coffee. But, of course, my plane was delayed. I got off the plane at O’Hare at 5:12 pm to make a connection at 5:15. Now I’m a large man these days but I did manage to run from the C concourse to the B in about 6 minutes – about 2 minutes too late to get on that plane. And all the flights to Seattle that night? Full. So my lovely wife gets to spend a night alone in Seattle while I sleep alone in our bed in Chicago. Nice. Then it’s up at 4am to catch a 6am to Denver, to catch a flight to L.A., and then off to Seattle.

As I type this I’m sitting on a plane in Washington, D.C.. I’ve been on this plane for over 2 hours but we haven’t actually done anything except take a tour around the tarmac and then head back to the gate for more fuel. Awesome. We spent so much time waiting for “de-icing” that we had to go back. Everybody’s being pretty jackassian about it. The lady next too me said, with no warning, “Hey mister, your going to need to move that arm of yours!” My elbow was touching her, you see.

Here’s the breakdown of yesterday’s commute to work:

10:00am get in car to go to airport in Seattle, WA.

12:30pm get on plane.

8:30pm (EST) get off plane.

10:30pm get on plane that has been delayed for an hour.

2:00am plane actually leaves the ground.

3:40am plane lands in Jacksonville, FL.

4:30am I get to my hotel.

Tomorrow I’ll post some pictures of the wedding – I’m going to bed.

Be the worst

Fri, 19 Jan 2007 03:13:00 +0000

So for about 2 months I’ve been working on this project with Clint Bishop and Nick Drew. Nick was the Project Manager/Tester/Technical lead and build guy. Clint and I were developers. Now that this project is being handed off to a new team (headed by Obie Fernandez), Clint and I have been working with developers other than each other. This is always a tricky thing because you have to justify the code you’ve written while working to make it better.

Even more interesting is that Clint and I are around the same level of Ruby/Rails experience while Obie and Zed (Shaw) are superstars. Not only do they have a long history in the Object Oriented field but they have been doing Ruby professionally for a while now.

Now I’ve been on a Rails project before and I’ve been messing with RoR for more than a year so I’m no chump, but I’m definitely behind the new guys. I was working with Zed for a few days last week and he’s just crazy fast. First of all, he has this behemoth HP laptop that has a tricked-out Linux distro with a custom window manager. Yep, he found a small window manager, got the source code, and tweaked it out. Also he’s a Vim dude. When he works it’s kind of an amazing montage of fast keystrokes, flipping through virtual desktops, and code appearing at an alarmingly fast rate. Then when we switch over to me driving, I fumble around in TextMate whilst he does a good job of not shouting “Faster!” ‘Cause, lets face it, he would have every right to.

A fair number of wise men advise programmers to find situations where you are the worst programmer in the room because that way you learn better. This is true – I’m learning a lot. But it’s a bit rough on your ego.

Now I shouldn’t feel bad, there’s probably only a handful of Ruby devs out there who wouldn’t feel nervous about their abilities after working with Obie and Zed, but its hard not to. It’s like when I was a first year teacher: I worked my ass off showing up early, staying late, obsessively planning my lesions only to see some 10 year vet teacher do it better and with less effort. It’ll come – I’ll get better. But until then I’ll just have to keep asking questions and saying affirmative phrases.

How did I get here?

Thu, 18 Jan 2007 01:18:00 +0000

So there’s only one more week left before this project goes live. But how did it start? Well, oddly enough, with arts and crafts.

See those faces in the above photo? I, and a bunch of professional consultants, went to Walgreen’s, purchased fashion magazines, then cut out and pasted pictures onto paper. Sounds kinda dumb, huh? Well it’s not – lemme try to justify: In our talks with the client it’s often helpful to focus on who’s going to use the software so that only features for your intended audience get in the code. Also, it helps keeps you focused on who you’re developing for. If everyone agrees that, to pick an example, Disco Stu is going to drive the content creation of the site then you might want to think about what he would like and what would annoy him. Why call him “Disco Stu”? Because “20 something bar patron” is boring. And if you’re going to be spending a 1-2 weeks discussing what Disco Stu would want in a website, then its cool to have a photo and a summary description of him (and all the other potential users) on the wall.

The whole point of the planning weeks is to get down to some atomic stories that describe features (or part of a feature). These are written on card and stuck on a nearby wall like so:

Why not put them in a computer somewhere? Well we do, but I also like to have a “big visible display” so everyone who walks in the room can see how we’re doing. Cards on the left are Backlog, they move into the current week, when someone picks them up they are “in progress”, then they go into “dev complete”, and then to “signed off.” In this picture Ryan has stolen all are signed off card to enter them into the computer. I’d like to get them all back up because the whole point of the “big visible display” is so that anyone who walks into the room can see where we are in the project, who’s working on what (my name is attached with sticky note to one of the cards in the picture), and what needs to be done.

Here’s the work environment:

Two pair stations. Each has a mac mini hooked up to 2 monitors, 2 keyboards, and 2 mice. So yeah, we do pair programming. This post is already too long without adding my defense of pairing, so I’ll save that discussion for later. In the right corner of the picture you can see our build machine. We check into Subversion, Cruise Control runs all the unit tests (using RSpec), then Cruise runs the functional/integration tests (using Selenium Remote Control to open up browser and click around), then if all tests pass it uses Capistrano to push the latest code to a place where people in the company can see and play around with the app. If the tests fail we get electric shocks.

Mechanical Turk can be beaten

Wed, 17 Jan 2007 00:22:00 +0000

Btw, here’s where I’m staying while on this gig:

See that boxy hotel, just right of center? That, my friends, is the mighty Sea Turtle Inn. And by mighty I mean: “Don’t stay there.” Nice location, crappy rooms.

Yesterday I was laid low by the perils of Amazon’s new RESTYSOAPY (because everyone wants their REST to be more SOAP-like) API, but today I have triumphed. I was awful ‘fraid Uber dev dudes Zed Shaw and Obie Fernandez would find a 2 second solution to the problem that had plagued me all day. They weren’t here yesterday and I was posative they’d come in and spot the problem all flash-like whilst snickering at my code. But of course I was just feeling self-conscious. Obie and I spent a few hours trying to put xml into an url without success until we noticed a line in the doc saying you can call a get OR a post. Oh. All I had to do was do a post with the xml in the body and … I ran into another problem.

The xml wasn’t valid without a whole bunch of stuff in the root element of the question. It should look like this:

<QuestionForm xmlns="http://mechanicalturk.amazonaws.com/  
    AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd"   
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
  xsi:schemaLocation="http://mechanicalturk.amazonaws.com/  
    AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd     
    http://mechanicalturk.amazonaws.com/  
    AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">

Which is nowhere I could find on the Amazon site. Obie and I actually found a website that let us enter question into a web form and then it produced the xml for us! Awesome. Now I can’t find it again. The Hell? It was at Awszone.com and I swear it wasn’t there yesterday and it isn’t there now. Maybe it’ll be back. I hope so because it was friggin’ cool. Once we saw all the stuff we were missing it was cake to stick it in to our xml and post a HIT (for Human Intelligence Task). Now all we gotta do is figure out how to get our answer back.

Hey, I found it again. It’s not linked to very well on the homepage of the site. Anyway, here it is.

The End to End Example

Tue, 16 Jan 2007 05:29:00 +0000

So this project I’m on is gonna be all sorts of User-Driven, but we don’t want users posting just anything (like say pictures of 9/11 on a page about a Restaurant). But how do you know a pic is “appropriate”? Computers just ain’t smart enough to do your job for you when it comes to images. And screening all those photos for badness would take an army of dudes.

Well our friends at Amazon are willing to put you in touch with those dudes. For cheap. Their Mechanical Turk service lets you pragmatically post odd jobs to their vast electronic job board. So now you can take a photo on you site, then post a job saying “Hey look at this image and tell me if it’s really Abraham Lincoln or just more boobies and I’ll give you 5 cents.” Then somebody picks up the job and does it. You can even review their work (spot check) to see how they’re doing. Or offer the same job to 5 people and pay 1 cent apiece and see if they all agree. Awesome.

Except that it’s not so easy to use. There’s 2 problems:

One – They wanna do both SOAP and REST. So their REST API looks kinda SOAPY. With REST I should just call a http get on a URL with some params, right? Sorta. The job needs to be formatted in xml. And they want you to use URL escaping but not xml escaping. So when I use Ruby’s built in escapper: CGI.escape it escapes both url and xml reserved characters. Fine. So then I do the escaping manually and… Ruby’s URI parser won’t see it as a good URL (which is kinda good, because without xml escaping it’s what we call “unsafe”). And now I’m stuck. All I get back is a response that tells me the job is invalid. But I don’t know how it’s really supposed to look.

Two – Which leads me to my second problem with the service: Their documentation doesn’t have a good End to End example. Sure there’s some trivial ones. I found an example, in Ruby no less, that showed me how to do a REST call to query my account balance. A good start. But what I really need is a walk through of some hard problem that takes me all the way up to exactly what the URL should look like in said hard problem. Because right now I’m just guessing about the escaping being the issue. Maybe it’s something else crazy easy that I’m missing. More I think about it, that’s probably it.

My Apprenticeship - Wednesday, August 18, 2004

Wed, 18 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary. This is the last post of the series.

Last day of XPAU and I’m pretty burnt. I’m typing this from the airport in Calgary where they inform me that they are running an hour behind. This is bad because my connecting flight in Las Vegas is an hour and 20 minutes after I land. So I’ll have 20 minutes to catch my flight, assuming they can keep to their prediction. I don’t have high hopes. I might have to miss work tomorrow.

Which would kinda suck because I need to figure out what’s going on this year. With Rylander and Miller gone, I’m going to need some time to plan with the new guys.

Today I slept in and skipped the morning stuff. Then I hung out in the FitFest room, writing code, and chatting with Micah, David, and Paul. Ken stopped by later and we said our goodbyes. I hope I see him sometime soon. The last address was on the history of the waterfall and agile. The presenter did a good job showing how the originator of waterfall was actually arguing against waterfall and for iterative development. And how iterative development actually has been going for 30-40 years. But it has been marginalized and contained. The hope is that the Agile movement can kill the waterfall once and for all. But, who knows, 40 years from now we might be just another slide in a presentation.

Me and the OM crew went out for one last bite after the conference wrapped. Hard to say goodbye to those guys. Paul and David live in the city so there’s a chance I’ll still see them, but Micah’s way, way out in the ‘burbs so that’ll be tough. Nothing a little effort can’t overcome.

Is this my last entry? It just might be, the apprenticeship is over. Thanks for reading.

I was pretty exhausted and down when I wrote those words. Setting up for a year of teaching takes a lot of work and I had done nothing. I was facing no sleep followed by a mad scramble to get ready for new students.

So what did happen? At this point I had already turned in my code submission to ThoughtWorks and when they didn’t get back to me after a few weeks I thought it was all over. What I didn’t know is that ThoughtWorks is known in the field for having a very long interview process. I had already talked to 2 different people and submitted code so I thought they had enough to make a decision. Sometime near the middle of September they called me in for a day of tests and interviewing. I got job a week or so after that and started in early October. Which meant that I had to leave school with the year just started, but my high school was cool about it and didn’t fine me or suspend my license (which they had the legal right to do).

I guess I could spend some time wishing I had continued this blog while working at ThoughtWorks, but instead I’m just going to be pleased that I managed to document so much of an important time in my life. There were days when I didn’t really feel like blogging after a day of getting my brain crushed by code but I’m glad I stuck with it as long as I did.

I still see Micah, David, and Paul fairly often, btw.

My Apprenticeship - Tuesday, August 17, 2004

Tue, 17 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Got up late, hustled downstairs to get some breakfast, only too discover that I didn’t care much for either of the Keynotes that morning. Some Microsoft guy was gonna talk about Visual Studio and M.S. agility. Hmm. I had already chatted with the guys at the Microsoft booth about the new Visual Studio and here’s the scoop: The will be unit testing integrated, but it will be Microsoft’s own version. They did say that V.S. will let other products add on and integrate on their own – but they couldn’t show me that. I will be interested to see if they will allow Nunit to integrate as seamlessly as IntelliJ and Eclipse do. So I wasn’t real interested in that. The other talk was about the future of C++. I’m still trying to avoid C++ until I have to acknowledge its presence, so I avoided that talk and hung out in the FitFest room. And had a good time writing some tests/code/test/code with a Swedish? guy named Carsten. When you carry a torch you can see the Wumpus for many rooms away, but he can see you. He will wake from his slumber and chase you until you are gone from his line of sight. The big problem was that ‘moveWumpus’ was private and only accessed from a method that had to do with arrows. So it took us a while to figure out why this was – the Wumpus wakes up when you shoot an arrow and miss. He then moves to a random room. So we stumbled around with various means of integrating this, but eventually ended up following the already established pattern of having a Boolean variable that is checked by the ‘evaluate’ function. So we checked ‘hasTorch’ in the evaluate method and then wrote an algorithm that looked along the lines of sight. It worked, but it is es muy crappisimo. First the line of sight business is duplicated in the shoot arrow function so we really should pull it out and abstract it. And the WumpusGame class has way too much going on in it. And the commands really suck (the are methods that shoot arrows, and magic arrows, but no way to access those command while playing the game). Unfortunately, both Carsten and I had to go before we could do this refactoring.

I talked to Micah about this latter and he had an interesting point. Even though the code was ugly, we had added way more functionality than last year (where the code size tripled and no new acceptance tests passed). And also the ugly code emphasized the need for sharing knowledge of the system by pairing and switching pairs. Carsten and I sat down and had no idea how anything worked. In order to add some functionality we had to poke around the code for 45 minutes. And then, because we had to leave and couldn’t work with other people, our knowledge of what needs to be fixed and how to fix it walked out the door with us. When we came back later, things had sufficiently changed so that we couldn’t be sure that our refactoring was valid anymore.

Later I sat in on a panel called ‘Is XP relevant?’ Which was mostly an excuse to start a fight. Bob started one by claiming that XP was irrelevant. Lots of people objected to the name ‘Extreme Programming’ because it scares people and others objected to getting rid of it because they think the agile is too watered down a concept too support. There was the same general disagreement about how much it is our job as developers to be polite and sell XP or Agile or whatever. I don’t like selling, but I don’t want to code in a waterfall company.

The big end of the Conference banquette was probably more loud music and dancing than most programmers are comfortable with, so when that started up most heading for the door. Before we cleared the room, there was the announcement of the next year’s conference name: ‘Agile United’ I guess XP is too scary to have in the title. The counter argument is that Scrum and RUP aren’t in the title so why should XP be privileged? Bob has worries that somebody might start up a conference with XPish title and all the unification we’re seeking will be lost.

Bob’s after dinner talk had to do with astronomy – surprise! He talked about how Kepler had a beautiful theory regarding the motion of the planets, but when he looked at the data he found his theory was wrong. So Kepler adapted. His idea was that XP is a beautiful idea, but we must keep looking at the data to weed out the bad parts.

For those who didn’t want to dance, there was charity poker. 10 bucks Canadian gets you $10,000 funny moneys. Then you could play blackjack or poker and for every $10,000 you turned in you go a raffle ticket. I ended up playing pretty well, got lots of raffle tickets, then Micah’s mom gave me her raffle tickets so I won two books. One was a Microsoft ASP book which I gave away, the other was ‘User Stories Applied’ by Mike Cohn. I saw mike Cohen speak on the Managing Agile Projects panel, and he seemed pretty cool (and like he might be able to break me in two) so I’m gonna read it. Not just because I fear for my safety, but also because a number of people said it was good stuff.

Bob is famous for sticking astronomy into any and all talks he gives. At this year’s Rails Conf he didn’t even try to integrate it into the talk. 10 minutes of entertaining stuff about the age of the solar system and then a bit about Smalltalk.

At some point this evening I talked to a bunch of people from ThoughtWorks, including my recruiter Sonia. I had met her though a reader of this here blog a week or 2 back but I hadn’t mentioned it because nothing was for sure. The ThoughtWorks people seemed pretty cool and everyone at Object Mentor said it was a good place to work. I think at some point this evening I made my pitch to Micah for a job at Object Mentor saying that I could be the IT guy for OM while I learned more and could slowly work my way into real work. Ultimately they declined my offer but were very nice about it and said that they just were not hiring at that moment but maybe in 6 months or so we could talk again. I was a bit crushed as I thought working at ThoughtWorks was a bit of a long shot. I had done a few other interviews and most companies were pretty put off by my lack of experience. So with XPAU just about over and my apprenticeship done it was looking like I’d have to teach another year of high school physics. Maybe this was all just a fun/stressful summer adventure.

My Apprenticeship - Sunday, August 15, 2004

Sun, 15 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Every time I think I understand FitNesse I’m wrong. After a day of using it for Acceptance Testing I know a lot more about it, but I’m going to assume there’s more. But this is jumping to the end, I’ll back up.

At the beginning of ‘XP for a Day’ Micah and James (the teachers) split the groups up and we started the planning game. James, the customer, had a bunch of stories written on 3X5 cards that he wanted implemented in the game. For example: One was a magic arrow that you could fire that would turn if it hit a wall (you specify the direction it will turn when you fire it). Another was a GUI for the game. And another was shooting regular arrows that would kill the Wumpus if they passed through the room he was in. Another was a randomly generated map. And so on. After James introduced the cards the time had come to assign numbers to them. This is one of the tricker parts for the first time XPer. The numbers are unit-less and do not correspond to a set amount of time. But the are a measure of how much time the task will take. Huh? Lemme ‘splain: Take all the cards and arrange them from easy to hard. Some are the same level so they will be a pile of easy, a pile of hard and a few piles in between. Now let’s call the features in the easy pile ones. What does one means? It doesn’t. But now we have a reference. All the other cards are defined in terms of the easiest ones. Is this task two times more difficult or three times more difficult than the simple ones? Unfortunately, since most programmers have a background of spending lots of time on design we spent too much time arguing over the points on each card. But it was a good learning experience. Then we guessed how many point we thought we could do in one iteration (1.5 hours in this mini-XP version). My group had no idea, but everybody else said 4 and we went along with the crowd. How many did we finish? 1 point. And here’s where I take a bit of issue with the class. Learning FitNesse is a day in and of itself. None of us were that familiar with it (I had used it in the TDD class I took, but some problems still were beyond me) and so we spent most of our iteration trying to figure out how to get the acceptance tests to work. Leaving precious little time to code.

Which is everybody’s biggest fear about TDD – that the test will slow you down. And I think one iteration will always be sacrificed to figuring out FitNesse or Junit (which we used too, but most were familiar with it) or whatever. The second iteration went much better. Now that we knew how to set up fixtures and get them to work, we were able to hit our target velocity much better. My personal feeling is that this needed to be two days if FitNesse will be new to most of the student (as I suspect it will be). If it has to be one day, then maybe unit test will have to be enough. Regardless, I learned a bunch and felt the class was a qualified success.

The problem with having students work on real things is that often they get hung up on something that shouldn’t be the focus of the class. I had this one week long physics project where the students would plan how to launch a spaceship and have it reach Mars. The first step is drawing a scale model of the solar system out to Mars. Most of the kids spent the better part of the first day just trying to figure out a scale so you can fit all the orbits on the paper. I had foolishly assumed that such a task would take a few minutes because it was easy for me.

My Apprenticeship - Saturday, August 14, 2004

Sat, 14 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Travel day today. Flew to Phoenix, AZ (4 hours) and then to Calgary (3 hours) plus 1 hour for lateness and layovers is an 8 hour trip that should have taken 3 (Chi to Cal). And there was a fun little dash from gate A11 to B7 (which are much farther apart than you might guess using your knowledge of the alphabet) because my first flight landed late.

Check in at the Hyatt Regency was easy and gorgeous. The conference space is really nice too. Micah, Paul, David, James, Justin (Micah’s younger brother), Bob, and I spent some of Saturday night setting up the ‘XP for a Day’ Tutorial space. There’s gonna be 3 teams of 4 (2 pairs each) working on ‘Hunt the Wumpus.’ So we needed to link all the computers up, set up FitNesse, and get CVS working. James has been nice enough to let me sit in on the tutorial, so I’m excited to see how much functionality we can add to ‘Hunt the Wumpus.’ Which is an old text-based game from the late 70s early 80s. You walk around a cave trying to shoot the Wumpus with an arrow. But if you go into to room with the Wumpus, he eats you. And there’s pits to kill you and bats to randomly transport you. Some functionality has already been done, so we’re going keep the project going.

Later we ate at the James Joyce bar and Bob talked about almost killing himself with carbon tetra chloride, high voltage transformers, and all sorts of other experiments he did when a child. I told some physics teacher stories. Ron Jeffries talked about the perils of rainwater management. We tried to get David to do some magic tricks (he was a magician in another life) but he politely declined.

Yeah, it was a pretty heady atmosphere at the eXtreme Programing Agile United conference. Bob knows everyone in the community so hanging out with him means you get to make a fool of yourself in front of some heavyweights. Ron Jeffries was tons of fun chat with – that guy has some stories to tell and man can he tell em’.

My Apprenticeship - Friday, August 13, 2004

Fri, 13 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Last day in Gurnee. The Object Mentors decided that pizza wasn’t enough for my birthday, so they took me out to one of those fancy Japanese steakhouses. Ya know, I’ve never actually been to one of those knife-flying, death-defying, make-it-all-on-a-grill-while-you-watch restaurants. A good time was had by all.

It’s been a good summer. I’ve learned tons about class design principles, patterns, pair programming, test driven design, and a whole bunch of intangibles. Without a doubt the coolest thing about working at OM was being able to lean over and ask David, or Micah, or Paul, or James, or – Look, everybody sitting next to me had great ideas about programming and, as cool as all their classes are, working with great programmers is a much better way to learn.

Speaking of Object Mentor’s classes, I’ve been meaning offer up my critique – from a teacher’s point of view. What I like, and what most non-teachers usually screw up, is the commitment to lab work. If there’s a lot of material to cover you can seem to cover it quicker with more lecture and less lab, but that’s a big mistake. Just because you throw a slide up on the screen and yap about it does not mean that anybody had any idea what’s going on. In my experience, 40-50% of the class are thinking about something besides what I’m talking about at any given moment. So it’s crucial to have the students work out the presented ideas in the lab. First, because it forces them to talk to each other and fill in the gaps where somebody was distracted by thoughts of girls, or candy, or shiny objects. And second, because if a student knows they’re probably going to put the presented ideas into action they pay just a bit closer attention. The lab, in this case, is writing some code. Because OM is XP, they do all code writing in pairs. Which is very cool. One person can get stuck, but two can tackle ideas that would confound either individually. So the classes don’t bog down when a few people are way behind and others have zoomed ahead. The groups tend to keep everybody moving along at the same pace (well not exactly, but it’s much better). Which is probably a good argument for pair programming in the non-classroom: Nobody gets lost. I tell ya, I miss it when I have to work by myself. There’s nobody there to keep me from doing something stupid so I waste lots of time running down a blind alley when a pair would look over my shoulder and say ‘What the hell are you doing that for?’

Well I gotta pack for XPAU – next post will be from Canada!

There are many times in life where you can go faster short term but you have to pay more on the back end. Sometimes it’s worth it (startups coming to mind), but I worry that once you’ve compromised your ideals once it’s hard to know where to stop.

My Apprenticeship - Thursday, August 12, 2004

Thu, 12 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

My Birthday today and the guys at OM were kind enough to buy me some pizza. Later on tonight I’m going to have a quiet little get-together at my favorite restaurant: Club Lucky.

Tomorrow is my last day at Object Mentor’s Gurnee location. Saturday I’ll be heading up to Canada and XPAU. Paul (the other OM apprentice) and I will be sharing a room at the Hyatt Regency. But here’s the fun part: The last day of XPAU is Wednesday and my job as high school physics teacher starts back up on Thursday. So I’ll be catching an 8:30pm flight from Calgary on Wednesday that arrives in Chicago (via Las Vegas) at 5am Thursday morning. Then it’s a cab ride to work and meetings that start at 7:30am. Let’s hope some sleep happens on the plane.

I’m pretty excited about XPAU, although I don’t know what I’ll be doing exactly. Basically Paul and I are Micah’s goon squad and any person or technology that gets in his way will be dealt with shortly. However, Lance mentioned that he might also need some help with a project or two he’s working on so I’ll probably be pretty busy. In between duties for OM I’ll be attending as many non-tutorials as possible. And trying to network (shudder).

My good friend Ken Hlebek (we went to high school together, fer cryin’ out loud) will be attending and presenting. Something called ‘Being Extreme with Eclipse.’ Ken is the guy who was kind enough to introduce me to Object Mentor and, since he lives in Oakland, CA., I don’t get to see him that often. Even when he does manage to get back to the Chicago area, he’s inundated by visits from and to family (big, big, big family) so I still don’t see him. Should be fun to catch up.

Weird how this all started: I get frustrated with teaching so I call an old high school friend and he puts me in contact with a guy he’s met once or twice. How this all leads to a new career seems improbably at best. But it does prove a point I’ve been trying to make about private schools forever: You’re not paying for a better education, you’re paying for the friends and connections your kid will make at the pricey school. Most people are shocked to learn that private school teachers make much less than their public school counterparts. Even at high schools that cost 30K per year. Almost all of them leave as soon as they can for a public school. So why doesn’t that experience and talent turn a private school into a terrible place? Well, 2 big reasons: First, people who spend 30K per year on education are generally smart, have raised their kids to value education, and care about what happens in the school they are paying so much for. And second, if you go to school with the children of such parents you will have connections to important people for the rest of your life and they will pressure you to do better in school. As a teacher, I’ve seen the same kid do well in a class of students who cared about school and then do terrible in a class that was indifferent. We all want to believe that we’re beautiful snowflakes of individuality but the truth is that the people around you influence your progress way more than we’d like to believe. I had a friend whom I could call to help me out and that’s how this all started. Now it’s true that I still had to do the work when I got the opportunity but without that opportunity I would have probably had to face years of night classes to get a degree in CS. And faced with that colossal amount of work with no certain outcome there’s a pretty good chance I would have just started showing more movies and phoning it in as a teacher.

My Apprenticeship - Wednesday, August 11, 2004

Wed, 11 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

I successfully broke the connection between the ChessBoard and the Pawn and the King. Pawn has its own PawnChessBoardIntereface and the king has KingChessBoardInterface. Each of these interfaces are pretty thin, so I’m back in a more comfortable place, design wise. I’ve also figured out a way to keep the rest of the pieces from needing to know about the ChessBoard. Instead of creating a FindAllLegalMoves method for each piece object, I’m going to look at the type of check the king is in. The king can be attacked straight on, diagonally, or it’s a knight attack. If it’s a Knight attack, this particular attack can’t be blocked so I don’t have to worry about trying. With diagonal or straight attacks I can find the squares between the king and the attack and then check to see if any of the king’s men can move to one of those squares with my, already created, isItALegalMove method. But I still have to create a method to find those squares between the threat and threatened.

David left his laptop power adaptor at home, so he mentioned a trip to CDW to get a spare. Paul, Micah, and I immediately perked up and jumped on that bandwagon. Micah’s wife called a little on and I overhead this little snippet of conversation: “Well I’ve already got lunch plans at, uhh, CDW.” We are nerds.

David is working on dot Net (he’s to be Object Mentor’s dot Net expert) Asp now. Asp is a scripting language, like PHP, but it’s Microsoft’s baby. Visual Studio has this Asp GUI (Graphical User Interface) creation tool which is WYSIWYG (What You See Is What You Get). But, as with most WYSIWYG stuff, you only sorta get what you saw in the tool. The idea sounds pretty cool: by dragging and dropping you can create a web-based GUI in a few minutes. And what it mostly does for you is give you a good start – there’s a bunch of manual code editing after the tool finishes. So David is messing around with it and he keeps checking the changes he’s made in Internet Explorer. Then he decides to check the results in Mozilla (an alternate browser): Guess what he sees?

Nothing. Absolutely nothing. And the stuff he’s asking Mozzila to do is totally stuff Mozilla can do (little boxes where you can type in data: A web form), but Microsoft has rigged it so that its product won’t work with competing browsers. Micah tried to open up the page with Safari, but still no go. He thinks there’s a way to get it to work on other browsers, but the point is, that by default and design, Asp has way decreased compatibility on non Microsoft products. Not cool, man.

As I was new to computing this willful disregard for other browsers was shocking to me. Now I just sort of accept it. Many of my readers probably develop web sites and know this, but I would estimate that supporting IE6 and IE7’s non standard behavior eats up 10-20% of any development team’s capacity on average. There may be weeks where you are only burning 5% on IE but then some huge hairy IE6 bug will come rolling in and you’ll lose a week of a developer’s time. Microsoft will tell you that if their browsers strictly interpreted html, most of the internet would be broken. They’re probably right: There are a lot of old webpages out there that have some busted html that only works in IE. But supporting them means making crazy problems for people who want to follow the spec.

My Apprenticeship - Tuesday, August 10, 2004

Tue, 10 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Well, I got the king’s weirdo move logic straightened out. And the pawn’s. But now I’m looking at my ChessBoard class and I’m unhappy. ChessBoard does a lot stuff and I’m beginning to suspect that all the chess pieces are going to have to know about it. Why, you ask? Lemme tell ya: In order to figure out if there is a checkmate I have to first figure out if the king is in danger. No problem: did it already. But checkmate means that the king not only is in danger but that there is no way to get it out of danger on the next move. One way to get out of check would be to move the king. So I need a findAllLegalMoves method in the king class. But to find all the legal moves, the king will have to know what pieces are on the board. Another way to get out of check is to take the piece that is threatening the king. I got that working earlier today. But the last way to get out of check is the killer. One of the king’s men can shield the king by getting between it and the threat. So now I need to look at all the king’s men and see if they can move into a blocking space. Of course, this means that all the piece objects need a findAllLegalMoves method and to do that they need to know about the board.

A UML diagram of my chess program would now have lots and lots of arrows heading towards the ChessBoard class. Which is terrible design. A small change in the ChessBoard would now mean that I have to change all of the chess pieces. How to break up this dependency nightmare? More interfaces acting as firewalls between my objects. Micah suggested I need to look at which methods each piece uses from the ChessBoard and create a different interface for each. So the king and pawn might each have their own interface with the ChessBoard, but the rest of the pieces would have a different and thinner interface (less methods). That’s a pretty big bit of refactoring so I better have at it.

Yeah. The chessboard stuff again. I find it kind of sweet that distant-past-Jake would spend all this time trying to arrange the dependencies so as to make the classes loosely coupled. But it’s also kinda ridiculous. Maybe a game like Go would have been a better candidate as its pieces don’t need to know too much but Chess, with it’s complicated rules and numerous corner cases, is very hard to tackle. Chess is like a business that’s been running for a few hundred years – any interesting problem is some sort of crazy exception to the general rules. So it was a good portent of things to come but a rather frustrating last project at Object Mentor.

My Apprenticeship - Monday, August 9, 2004

Mon, 09 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

XPAU is only a week away and most of the things that need to be shipped have been shipped. So Monday was the calm before the storm. Micah had a hundred or so neglected emails to answer and some odds-and-end work to do. David and Paul were pairing on the dot Net FitNesse project (making FitNesse work with C#). And I have been asked (dared) to write a chess program by Micah.

Before I get into the intricate parts of chess software, I should mention David’s correction of my previous post. On the subject of acceptance testing vs. unit testing, he says that it’s important to point out that the customer owns the ATs and developers own the UTs. A developer can have loads of passing UTs, but it’s the ATs that the customer ultimately cares about.

On the subject of Chess, I’m finding that just getting the computer to moderate a game between two humans is a more interesting problem than I thought. The main culprits are the pawn and the king. Here’s a list of king behavior the computer must enforce: kings can move one square in any direction (including diagonals), they cannot move into check (a square that is threatened by an opposing piece), they can castle ( king side castle: move two square to the right and have the king side rook move two square to the left. Queen side castle: king moves two squares to the left and the queen side rook ends up on its immediate right.) but only if the king hasn’t moved yet, the rook hasn’t moved yet, there are no pieces in the way, the king isn’t castling out of check, or through a space that would be check, or into check. A little bit conditional, eh? I’ve got all that programming done except for the moving the rook part. In my design, pieces can tell the board object if a given move is a legal move and then the board can use that information to decide whether to move the piece. The problem is that if the king castles, I have to have some way of telling the rook to move. I could program that into the board object, but then some of the king’s logic is stored in the board object. This is very bad programming form. Latter on if I, or somebody else, wants to create some sort of chess variant where the king moves differently this person won’t think to look for part of the information about the king in the board object. So if they try to change how castling works, the board will keep trying to move the rook and that could cause all sorts of hard to find bugs. And I would have to go sit in the bad programmer corner. I’m thinking about creating a ‘move’ command that can be issued to the pieces which, in most cases, would do nothing but in the case of the king and the pawn (which can turn into another piece if it reaches the back rank and has this weird way of taking another pawn without landing on its square called ‘en passan’) would do some additional things. What bugs me about this is first that it’s dummy command for most of the pieces and that it doesn’t really do what it says. ‘Move’ won’t make the pieces move (the board takes care of that), it’ll just get them to perform the actions of certain special moves. I need a better name.

Also there’s the problem of checkmate. Check isn’t so bad: I wrote an algorithm that looks at all the opposing pieces to see if they threaten a given square. Now checkmate might just seem like looking at all the possible moves of the king and seeing if they also put him in check, but it’s a little more tricky than that. There’s two other ways to get out of check besides moving out of the way: you can take the piece that is threatening your king, or you may move one of your pieces in front of the king to protect him. Both of which are interesting programming challenges.

Yes the Acceptance Tests should be owned by the customer. However, the developer often writes the Acceptance Tests because while the customer technically owns them he or she doesn’t have the technical ability to make sure the tests do what the customer wants. This can lead to trouble if, for example, the customer wants to make sure a user can be created, logged in, change their profile, and logout. A developer writing that test might be tempted to just go right at the database to create a user (which really doesn’t click through the app the way a customer would) and while the test would appear to be testing sign-up that page could be busted without the test failing. Getting a good Acceptance Test framework up and running, maintaining it, and making sure it tests what you care about is a daunting task and takes way more effort (and hardware) than most clients realize. The out of the box solutions that record clicks are either inflexible (you can’t do loops or have variables or assert what you want), proprietary, slow, or (usually) all three. And custom solutions take time and precious developer resources. In short, automated acceptance testing is hard and needs a fair amount of resources.

My Apprenticeship - Friday, August 6, 2004

Fri, 06 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Last day of the TDD and refactoring class today. We started with an intro to FitNesse, which is acceptance testing. The basic idea is that while unit tests are concerned with the inner workings of the classes and methods, acceptance testing is after more global behavior. An acceptance test tends to simulate the actual operation of the program. What you do is enter some data in a table that the user might enter. Now this table is in a wiki which is being served by FitNesse which may be running locally or on the company server. Then, when you push the button (on the wiki page) that sez ‘Test,’ FitNesse parses the wiki text and feeds the data from the table into your program. And then it checks the output against the table you created (on the wiki) with the expected output.

Which sounds a lot like a unit test – just applied to a larger scale. And it mostly is, but with the added benefit that it’s easier to write a test in a wiki than to learn java, or C#, or whatever. That means that the customer can write the acceptance tests and you can gets busy making them pass. Which is pretty cool.

The tick of Fitness is that you have to write a bunch of interface code between your natural language wiki tests and your code – In that way it’s kind of like cucumber. But anyone wishing to write Fitness/Cucumber tests has to learn to think a bit like a programmer because you can’t write just any sentence and have the magic computer turn it into a test. We programmers often forget how hard our job is until we see non-coders attempt to do what we do every day. Often a team will try to adopt an acceptance testing framework because the tests look so easy to write but that is deceptive – computers can’t really read natural language yet and so writing acceptance tests as a non programmer can be frustrating because the cucumber or selenium or fitnesse or whatever tests look like sentences but a slight deviation from the form produces a cryptic error message. Ultimately I think they are worthwhile but you have to have a serious commitment from the people writing the tests to learn how write the tests.

My Apprenticeship - Thursday, August 5, 2004

Thu, 05 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Well, knowing more about mocking objects would have helped a lot when I was writing my TDD Tic Tac Toe. Mocking Objects was, of course, one of the subjects of today’s class. The whole day was devoted to TDD (the first day we were refactoring existing sample code) but I was pretty familiar with most of it except for the mock object pattern. Which seems like a big part and just goes to show that sometimes it’s nice to have someone explain something to you even though you think you already understand it.

I’m working with a possible future intern: Matt. He’s at DePaul and might be helping OM out in the fall (he has to work around his class schedule). Matt’s been taking a bunch of non-computer classes this summer so he’s a little rusty in the programming department. And he keeps wanting to write code without tests. However, I remember my first day at OM when I couldn’t even get a simple bowling score program to work (yes, they try that one out on everyone). It’s hard pairing with somebody who knows a lot more than you. Can be intimidating.

Five years down the line I’m still learning something about testing every day. I’m giving a talk entitled “What’s the Right Level of Testing?” at Agile 2009 and Lone Star Ruby Conf – in it I discuss the difficulties of balancing testing, team moral, velocity, assurance, and code quality. After looking over my slides I have to say that most of the teams I’ve been on have gotten it wrong more often than right. Which is all the more interesting because I’ve been working at the test-friendly ThoughtWorks and Obtiva. Testing is hard (but seems deceptively simple) and it can easily be over or under done.

My Apprenticeship - Wednesday, August 4, 2004

Wed, 04 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Good news! With Jet Brains’ ‘Resharper’ plug-in installed, Visual Studio is way more fun. Not as cool as IntelliJ, but it’s getting there. As you might have guessed, the TDD and Refactoring class is being taught in C#. The first day was weird for me: I realized that a few months ago I would have been way overwhelmed but now I was following along with ease. I wasn’t some sort of huge revelation, it’s just that while James was writing a bunch of UML on the board I not only didn’t have that sinking feeling UML used to inspire but I knew what pattern he was going for before he finished. Cool. The course is shaping up like this: Basic refactoring and Unit tests today, tomorrow should be test-first programming with refactoring (always with refactoring), and I suspect we will get into FitNesse late tomorrow or Friday.

A few minutes ago I finished up my TDD re-write of Tic Tac Toe and I’m including a zipped version of the source code here (tests are included, of course) [Future Jake note: the files can be found on Github at http://github.com/jscruggs/tic-tac-toe/tree/master in the third_pass_tdd directory]. Feel free to open it up, run the tests, and give me some feedback. It’s a text-based game so don’t expect a gui or anything fancy. I use a minimax algorithm to make the computer choose the best move. If you want to know more about my history with the Tic Tac Toe program, you should look at my may 2004 post.

How do I feel about writing TTT test-first verses without-a-net? I gotta say that the tests make everything more sane. With TDD, I could focus on one problem at a time without having to be driven crazy by weird bugs and mountains of suspect code. During my first non-test version of minimax I had this bug where when I asked the computer to move it filled in all 9 of the spaces. I spent days staring at code before I realized that I was passing a reference to the board instead of a value (so the board got changed as the computer tried checked of the possible moves). Of course I didn’t find this bug until I had written enough code to make it run, so I was stuck looking at everything. When I wrote the minimax test-first, I focused on just the minimax method and the test. That’s it ‘ any problems had to be somewhere in those lines of code. Which made it lots easier.

Now you may object that having written TTT once, it was easier to write it a second time. Yes, but when I re-wrote TTT in Python, without tests, I still had tons of problems with bugs (see blog entries 6-14-04 through 6-17-04).

Once again I have posted my early bad code to github so you can see it in all its splendor. And that action gets at one of the reasons I’m re-visiting these posts: To point out that everyone came from somewhere. When I was getting started in teaching and programing I often made the mistake of lionizing those above me (without even thinking about it) instead of realizing that they were just people like me who hard worked hard to get better.

My Apprenticeship - Tuesday, August 3, 2004

Tue, 03 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Other than a little bit of time spent with Micah explaining Word Press to Bob, I was able to code all day. Whoo! Some hints from Micah yesterday and a lot of trial and error today led me to finally get the hang of testing methods that rely on user input from the keyboard. When the program is really running, I pass in a regular old keyboard InputStream (System.in) to my userInput method. However, in a test, I can load up a ByteArrayInputStream with all the values I want entered (separated by newlines, of course) and then every time a function I’m testing calls for a user input it gets my preset values instead. Which is pretty darn cool.

Of course this took a large chunk of the day to get it working exactly how I want. Java IO can be a wee bit tricky. The rest of my time was spent applying the finishing touches to my long-neglected TDD Tic Tac Toe game. Should be ready for prime-time tomorrow.

Speaking of tomorrow, I’ll be taking a class with James Grenning: Test Driven Design and Refactoring. Unit testing has been interesting, but I’m itching to see how FitNesse extends things. Since the paying customers want the class in C#, there’s a good chance that I’ll revisiting Microsoft Visual Studio. We gotta get Jet Brains’ add-on installed soon because life is unpleasant without refactoring tools

When I applied to ThoughtWorks I used the trick I learned above in my coding submission to test input from the command line. Later on when I got hired I asked to see what people said about my code and one of the comments was “Some of the things he tested would never fail.” First of all – the anonymous reviewer was right. One can reasonably assume that core Java IO will work and so it doesn’t have to be tested in your application. But second – it was pretty cool to see that I had tested way more than a ThoughtWorker thought was necessary.

My Apprenticeship - Monday, August 2, 2004

Mon, 02 Aug 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Word Press has been successfully installed on the Object Mentor servers and ‘ButUncleBob.com’ should be available for blogging in a few days. So head on over there sometime soon and pick a fight with Bob Martin about some XP practice or other.

The install, as usual, was a bit of a problem until we realized that since PHP and MySQL were installed separately, we had to run a program to make them talk to each other. Said program was downloaded, installed, and then everything went smoothly. Nice.

Micah has now officially charged me with making a chess program to compete with his creation. Baby steps, Micah. First I gotta get it to let two humans play each other. After some discussion I’ve given up on my ‘special moves class’ because it’s way more of a maintenance nightmare then the problem it’s supposed to solve (letting ChessPieces know about the ChessBoard). I also want to finish my TDD version of Tic Tac Toe. The Land of Linux was fun to visit, but it’s good to be back writing some tests and makin’ ‘em pass.

Yeah, I kinda stayed away from Linux for awhile after that first experience. It’s all summed up here in a nice and cute sorta way but there was a few weeks there of coming in every day and feeling like my operating system was out to get me. I literally knew nothing about using ‘Nix OS so every problem was hampered by the fact that even the simplest of things (like using ps and grep to find out what was running, or the ‘which’ command, or where command paths were stored, etc.) was a herculean task in the face of my limited abilities.

My Apprenticeship - Friday, July 30, 2004

Fri, 30 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

IntelliJ IDEA 4.5 just came out and there’s a been a flurry of emails about how it lets you analyze your code. The Martins (Micah and Bob) are trading barbs about whose code is more redundant. Paul looked some 10 year old code he was working on and it had hundreds of repetitions. I’ve only just installed 4.5, so I haven’t had time to try the new features. Dare I analyze my old projects?

The pawn is killing me. My chess program takes the pretty obvious tact of having a ChessPiece interface from which all the pieces inherit. Bishops, Knights, Rooks, don’t really need to know about the ChessBoard. You can just give them their move and they’ll tell you if it’s legal. The ChessBoard will do a check to see if any pieces are in the way of a move. But the Pawn – What a jerk! Pawns can move two spaces on their first move but not on their second. That’s not too bad – I’m using a state pattern to take care of that. But they attack on a diagonal if and only if there is an opposing piece for them to take upon that diagonal square. Also they can do this weird en passan thing if another pawn passes them by making a two square move, but only on the next turn. It’s crazy insane. So now I’m faced with the possibility of passing in not only a ChessBoard, but a LastMove to all the ChessPieces even though only pawn really needs it. Well, actually, the King also needs to know about the board for castling, but I’m trying to fight one war at a time. I’ve been thinking about creating a special moves class which would check for pawn diagonal attacking and en passan and other things. But that’s pretty ugly too as the pawn logic is going to be in a number of places. Hmm.

Of course all this is a little silly because the rules of chess haven’t changed in over 200 years are aren’t likely to change any time soon. I could, in theory, couple the code as tightly as I want. But the whole point of this exercise (for me) is to learn something about managing dependencies so I’m going to pretend that the rules may change.

Some sort of program that analyses the quality of your code? Is this the birthplace of metric_fu? Not really, just the beginning of a thought that really didn’t mature until much later.

I love that I’m trying to keep knowledge of the board away from the pieces. I really spent a bunch of time drawing UML diagrams and thinking about dependencies before writing any code. If nothing else, it was good practice thinking about design.

My Apprenticeship - Thursday, July 29, 2004

Thu, 29 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

So I was doing one last re-install of Word Press, just to make sure I understood everything that could go wrong – But apparently there was one more thing. I just couldn’t sign in from my laptop when I pointed it’s browser at the Linux box. I know I had the right login and password because it worked on the other machine. After spending too much time messing around in the Linux terminal, I went back to my laptop, deleted the cookies associated with that website, and everything was fine.

Cookies are a problem.

The chess program is proving to be an interesting problem. Everything wants to know about everything else. Must… Manage… Dependencies…

“Everything wants to know about everything else” I fight this battle every day. And I continue to fight cookies and other browser state tricks. State is the enemy.

My Apprenticeship - Wednesday, July 28, 2004

Wed, 28 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Not a whole lot going on today. I messed around with Word Press. Paul continues to remove the html from FitNesse. Which is more like storing the html inside methods so that the chance of a typo is reduced – if you have to constantly add in a bunch of cryptic strings then you probably will mistype somewhere which can cause hard to find problems. However, if you make a method that adds in a html tag, then either every instance of that tag works or fails. The IDE will tell you if you’ve entered the method correctly, thereby reducing the chance of a typo causing a troublesome error.

Susan and Ellen are pretty swamped under the pressure of getting XPAU up and running. David’s having a good time with his class, but they’re working him like a red headed stepchild.

I got to meet Micah’s wife Angelique and their cute-as-a-button son today. Luka responds to French, because Angelique was born in France. We all had a good time watching him toddle around OM. ‘Don’t put that in your mouth’ seems way cuter in French.

My Earthlink DSL is down at home. I sent the Earthlink crew a long email detailing all the things I tried. They sent me back a standard response that asks if I’ve tried shutting down the computer and modem and then restarting. Of course, I mentioned that I had already done this in the initial email. Sigh.

Earthlink? I forgot that I had DSL for awhile – it was terrible. Now I have RCN and it’s terrible. For awhile I had Comcast and it was terrible. There’s a theme here… Is moderately fast and reliable so hard to figure out? This morning I spent 10 minutes waiting for a 10 meg podcast to download. It still had ‘25 minutes’ to go when I had leave for my train.

I saw Luka a few months ago – He’s still very cute. And he has an iPod touch.

I want an iPod touch. Luka has a portable game machine/music player 10 times better than all the audio/video equipment I owned before the age of 25. Oh to be the child of a nerd with money.

My Apprenticeship - Tuesday, July 27, 2004

Tue, 27 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Today I worked on cleaning up Word Press so that it looked nicer and behaved better. Alex King was nice enough to hold a contest for the best Word Press skins and publish the results here:

www.alexking.org/index.php?content=software/wordpress/styles.php

For now, ‘But Uncle Bob,’ is using the ‘Dark Fire’ skin. I also spent a fair amount of the day figuring out the various features of the program so that I can teach Micah when he comes back. Once you get it working (and, if the server has already been set up for you, that is not too hard) WP is a nifty little program. Tomorrow I think I’m going to wipe the Linux box, reinstall the OS, and put WP back up because the first install was so crazy that I’m not completely sure I can repeat it without wasting a whole bunch of Micah’s time (Micah’s off in Philadelphia this week coaching some company called Primavera) .

Somewhere in the middle of all this, it has occurred to me that there is a fair amount of irony involved in blogging about trying to set up blogging software. But the bigger question is: now that I know how to get WP up and running will this blog become a Word Press blog?

Probably not.

Not that I hate the program, but I’m not interested in its advantages. Right now I type my post in MS Word, then do a little cutting and pasting via TextPad, and upload with an FTP or Web-based file manager. Upgrading to Word Press would allow people to post comments, or let me set up a list of users who are allowed to post, but I don’t much care about that. If somebody wants to comment they can email me. Also, I’m not sure about how long this blog will last after August 18th (the last day of XPAU and my last official day with OM). After that I’ll either have found a new job in software development (Interested in hiring me? Send an email and we’ll talk.) or I’ll be back teaching physics to teenagers.

David’s Advanced Object Oriented Design class is going well this week – one of his students speaks Portuguese and so does David! What are the odds? It’s a worldly class, other class members are from Italy and India. There was a little trouble the first day when they turned up their noses at the complementary Crispy Cream donuts. One student even when so far as to pejoratively call Crispy Cream’s ‘So very American.’ Look, you can take shots at our foreign policy, but insulting our donuts is asking for trouble.

Hostilities, thankfully, were avoided.

What? Still talking about WordPress? Boring.

It’s funny that I’m talking about entering the blogging world while a fair amount of my colleges have abandoned serious blogging for Twitter. In my quest to be continually 2 years behind the times I’ll be announcing my Twitter mash-up website soon. Via stone tablet.

But seriously folks, I am indeed on the Twitter where I am known as jakescruggs. I recently spent a bunch of time tweeting about the movie ‘Bolt.’ Seemed like a good idea at the time.

My Apprenticeship - Monday, July 26, 2004

Mon, 26 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Word Press is up and running! I had a bunch of ideas when I arrived, but before I got too far I said to my self: ‘Self, maybe you outta try starting Apache, MySQL, and run install.php from Mozilla just one more time’ Well, I wouldn’t be telling ya about it if it hadn’t worked. Of course when I say it worked, I mean that the install actually started but not that everything was fine from the get go. First I gave it the path that the wordpress folder was in, but not where the files were. So everything failed, but did so is a way that I could see that every place it was looking was missing a ‘/wordpress’ so I fixed that. Then I tried to look at it on another computer but I was getting a screwed up version of the site. What wordpress really wants when it asks for the location is something like this: “181.783.13.1/wordpress” Eventually that’ll be something a wee bit more user-friendly, but it’ll do for now. Finally the install went smooth. Excellent.

So why did this take forever? Here’s the Reader’s Digest version: Red Hat Linux automatically installs MySQL, PHP, and Apache so when I attempted to install XAMPP (which is a bundle of the three) things got a little nutty. Same problem when I tried to install MySQL, PHP, and Apache separately. But all this might have been fine if I had known that the way you’re really supposed to run install.php is not from the command line but by typing in install.php in a browser pointed at your sever (which, for me, would be http://localhost/wordpress/wp-admin/install.php). Over a week went by with me trying everything I could think of but it didn’t matter because every time I typed in ‘php install.php’ I got the same error telling me that: maybe MySQL isn’t running, maybe your server isn’t running, maybe you set up the password/username are wrong, or maybe PHP is configured wrong. Not one of those things actually turned out to be the problem, but I did learn lots about Linux, MySQL, PHP, and Apache in the process.

In response to my request for a side-project, Micah has suggested that I join the club and write a chess program. This may keep me busy for awhile.

The weekend in Michigan was fantastic. My former boss gave me ‘a bicycle built for two’ (are you thinking of the HAL 9000? I am) and it’s been rusting in my backyard for 9 months. On Friday, I took ‘er apart and crammed it into my Saturn (no easy feat, mind you. but nothing some WD-40, the wrong tools, and a whole bunch of swearing couldn’t fix). Father and son were able to put the tandem back together in Michigan and it’s a whole busload of fun! The brakes are a little dicey and the tires might be leaking, but everyone’s your friend on a tandem. Cars slow down to look, kids laugh and point, people wave and say nice, friendly things. Riding along the lakeshore with my wife was quite the time. Here’s a picture:

So I started the 5 minute install back on July 8th and now, a mere 17 days later, it is complete. By the way ButUncleBob.com (the blog I was trying to set up) had a good run from 2004 through 2006 but has since been retired/mothballed.

My Apprenticeship - Thursday, July 22, 2004

Thu, 22 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Finished up the patterns class today and I have to say that David did a pretty good job. With new material, not much teaching experience, and Paul and I firing questions at him he was able to get through the class rather easily. Sure there were some confusing slides and weird-ball patterns, but the cool thing about OM giving him this week to flesh out the course is that he can make note of the problems and then talk to Micah or Bob to straighten things out. Next week he does it for real with paying customers and I think he’ll do fine. I, however, need to write more code. This Word Press/Linux/MySQL/Apache/PHP adventure I’ve been on has been very informative but sorely lacking in full-on coding. I need to remember to ask Micah (or Bob, or David, or Paul) for some advice on a side project to make me a better programmer. If any of you are reading this and feel like emailing me an idea, feel free. Life without cool little software projects is less fun.

Tomorrow I’ll be heading up to Michigan to see my folks and hang out at the cottage. There are some pictures of my last trip here.

The cool little side project from Micah will turn out to be not so little.

My Apprenticeship - Wednesday, July 21, 2004

Wed, 21 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Our payroll UML diagram (from yesterday) had some needless complexity but, other than that, it was fine. Well, mostly fine.

Lots more patterns today. Many of the patterns boil down to the server-client relationship. Let’s say you have a laser scanner object that can scan a bar code and identify a product. There’s also going to be a sale object which gets a message from the scanner and then does some sort of sale stuff. Well now the scanner has to have code that knows about the sale and the sale might even have code that depends on the scanner. This system is now tightly coupled. But if you stick and interface between them, you can have them both know about the interface and not each other. Therefore swapping out the scanner for another one doesn’t affect the sale code. And changes to the sale don’t affect the scanner object. Patterns can be way oversimplified as the proper placement of interfaces. Too many and you’ve got needless complexity. Too few and the code is tightly coupled and hard to maintain. All these patterns have trade-offs so the big deal is not just knowing how to use them, but being able to figure out if they’re necessary/will help your project.

My project for the afternoon was, of course, banging my head against the Linux box. Apache keeps insisting that I don’t have permission to run the install.php program, but everybody has permission to use it. Hmm. At least Micah had some trouble when he tried to figure it out. I’m always a wee bit worried that the thing that’s been bugging me for days is only 30 seconds worth of work for a pro. So I went back to prowling through the httpd.conf file (a set-up file for the apache file server) for things to change or adjust. The day ended with Micah and me pretty stumped.

The big message I took away from Object Mentor that summer was that although patterns are cool and can help a lot, they add complexity. It’s a complexity/coupling trade-off. A message that has stayed with me. Just today I was looking at some code that could have been refactored to be more DRY, but way less intentional and I had to make a call as to wether the project would be better served by adhering to one principle or another. That’s the thing about writing code that is meant to be understood by humans: The hard part is not learning the language, patterns, or principles, but knowing when to use them and where.

My Apprenticeship - Tuesday, July 20, 2004

Tue, 20 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Second day of A.O.O.D. and I’m getting pretty good with UML. Today we discussed the various principles of software development: Single responsibility, Open/closed, Liskov, Interface segregation, and Dependency inversion (Which, all together, spells S.O.L.I.D. - you can learn things from XPAU mouse pads).

Paul an I spent an hour or so trying to design an system that could take in information about a bunch of different types of employees. Then, after we designed a system with enough interfaces and uses of inheritance to avoid violation of the principles, more user stories were added and we had to refactor. It was an initially frustrating, but ultimately fun project to take a system and make it do something new without, hopefully, coupling the classes together too much. I think we got it, but we’ll find out tomorrow morning if we did.

Bob said he liked my blog, especially the bit where Micah came back after a week, looked at out project and asked us what we thought we were supposed to do. Apparently this is a pretty common experience in software – after a few weeks the customer looks at what you’ve produced and either realizes that what you did isn’t what s/he asked, or what they asked for isn’t what they really want. One of the big points of XP is trying to make this conflict between intent and execution happen earlier rather than later. There’s always going to be changes to the system, but big ones need to happen early (changing the look and feel of the GUI – not so bad, changing a 2-D side scroller into a 3-D first person shooter? Little bit more difficult).

I still have one of those ‘S.O.L.I.D.’ mouse pads - I don’t really need it for my optical mouse but I have it. Also nice to have an Uncle Bob (Martin) sighting. People often ask me about all the Object Mentor employees I must know, but most of them were off teaching classes at the client site or doing agile/xp coaching during the summer of 2004. I mostly hung around with David, Paul, and Micah that summer and had shorter interactions with the rest. The same thing happened at ThoughtWorks were people would assume I knew this, that, or the other person who worked there while I did. But often person ‘X’ was in Texas while I was in New Jersey and our interaction was limited to possible bumping into each other at a company function.

My Apprenticeship - Monday, July 19, 2004

Mon, 19 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Today was the first day of my Advanced Object Oriented Design (AOOD) class. David’s teaching Paul and I in order to make sure he’s familiar with the material and can teach paying customers next week. After covering the usual X.P. verses Waterfall stuff and briefly touching on how good OO design can keep your code loosely coupled (in a tightly coupled design each class depends on several other classes, which in-turn depend on a few other classes and so on. The bad news is that changing one class means that you have to change every other class that depends on it. And since everybody is holding hands with everybody else, that could mean hundreds of changes for each modification. No me gusta. Loosely coupled code has objects that, through interfaces and clever structure, can be changed without having to track down bunches of dependencies) we moved on to some UML (Unified Modeling Language – a way to get a global view of the code you are about to write). Which is cool because although I’ve been looking at and trying to draw UML for a month or so now, I could really use some formal training. We used UML to design a state machine that strips code of its comments. In a few languages (mostly C derived) two slashes mean that the rest of the line is a comment and should not be read by the compiler. Or anything between a slash and a star and then another star and a slash is a comment. For example:

Not a comment // comment all the way ‘till the end ‘o the line

Not a comment /* comment */ not a comment

So we wrote out a bunch of UML and it was pretty interesting. I like all the cases where the object loops back to itself – I don’t know why I find that cool, I just do. After working on that SMC for a week or so I had a pretty good foundation in State Machines but it was fun to try out my ideas on something new. Later we had to make a UML diagram for a cash register. There were a number of user stories which we had to accommodate. Knowing something about teaching, it didn’t come as a great surprise that this exercise was supposed to be a bit of a failure. The were multiple dependencies and inversion violations and single responsibility problems as far as the eye could see. And this was after an hour’s work. But the point of the class is to get us to be able to use design patterns to eliminate (or at least minimize) these problems. It’s basically a patterns class. Patterns are the idea that many programming problems boil down to the same idea and that there are well thought out ways to deal with these common problems.

Oh, and I’ve got a potential solution to the Word Press problem. The install.php file needs to be run by the server. In other words, I need to request the file from Apache which knows to execute .php files. Interesting. This was nowhere in the installation notes, but I guess they probably assumed that anyone trying to set up a server would know how things are done with servers. I didn’t have much time today to try out this idea (which Micah came up with) but maybe tomorrow. Although, with the class this week I won’t have a lot of extra time.

Weird that I just had dinner with David (Chelimsky) and his lovely lady friend Flor and now I’m writing about him teaching me a class. David and I actually live in the same condo complex in Rogers Park. This happened more or less by accident. David currently works for Articulated Man and I for Obtiva -- we’ve never worked together professionally but have remained friends.

Did you see that part about how you’re not supposed to run the install.php file through the command line but by hitting it through a web-browser? I didn’t know at the time that Apache would just figure the rest out and the people who wrote the instruction manual probably didn’t think they had to point out something so basic. A common problem when I was getting into software was that no explanation was basic enough for the likes of me.

My Apprenticeship - Friday, July 16, 2004

Fri, 16 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Today I rode my bike into the city in search of a passport. City Hall: Big, confusing, lots of stairways leading to locked doors. First goal: Get a birth certificate. Long line, filled out misleading paperwork wrong, but accomplished. Second goal: Get passport. Not so fast, Mr. man, this office is closed inexplicably for an hour starting… Now. Kill time, eat pizza, come on back. I was waiting at the wrong place! Even thought the sign above the door clearly said ‘U.S. Passports.’ But the real passport place is upstairs in the same room where you apply for city jobs.

Silly of me.

For all those of you who like to complain about yer passport photo, I’d like to point out that you probably didn’t wear a bike helmet in rain and let your hair dry in a pizza parlor before you got your photo taken. Did ya? That taken care of, I paid the nice lady $160 ($85 for city and federal fees, $60 because I want the passport sometime this year, and $15 for the photo) and peddled my soggy self home.

I remember being pretty worked up about asking Object Mentor for a day off so I could get my passport. They were very cool about it. On the other hand I was working for free that summer. And by working I mean floundering. They were probably excited to have a day where they didn’t have to listen to me complain about a ‘5 minute install’ all day long.

My Apprenticeship - Thursday, July 15, 2004

Thu, 15 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

So I downloaded and installed Apache – which wasn’t too much trouble. But MySQL was a different story. When typing in the bizarre list of commands to install it, I made a slight typo which took a little while to track down. But, even worse, I ran into the same problem as before – MySQL would run but Word Press couldn’t find it. After a lot of messing about I finally figured out that Linux had installed MySQL automatically and that was causing conflicts. Which I couldn’t figure out how to resolve. Before I installed PHP I decided to look and see if it was on the hard drive. I found the documentation, and when I typed in PHP at the command line something happened. But I couldn’t find where the files where installed. If this were windows I would have some idea how to go about looking for things. Add/Remove programs can tell you what’s installed. The find function in Linux doesn’t seem to work as well as windows. This is all part of the big problem here: I don’t know Linux yet and so all my usual tricks do not apply. Thursday ended with Word Press’ ‘five minute install’ as yet to be completed. It’ll have to wait ‘till Monday because tomorrow I’m heading downtown to get a passport. In theory, you only need a birth certificate and a driver’s license to travel to Canada, but I have these visions of my flight taking off as I try to convince some large man that I really am a U.S. citizen. Best to be sure.

Turns out most Linux installs come with Apache, PHP, and MySQL – this supporting of common developer tools was weird to me coming from Windows where if it’s remotely useful it must have a heinous install and cost a lot of money.

Why was I going to Canada? Because that year’s XPAU (Extreme Programing Agile Universe (Now just called Agile as in ‘Agile 2009’ - have I mentioned that I’m speaking at it?)) was in Calgary that year. Object Mentor used to organize XPAU, so it was a pretty big deal for them. And me. I think this might have been the last year Object Mentor was running the show.

My Apprenticeship - Wednesday, July 14, 2004

Wed, 14 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

The problem with a ‘five minute install’ is that if you go beyond 5 minutes, you are screwed. A five minute install is doing a lot of things behind the scenes. If those hidden processes fail, then you don’t have anything to look at. The configuration file for Word Press is very simple: You tell it the database’s name, the user’s name, the user’s password, and where to look for the server. Once you’ve verified that the database does exist (by messing around in MySQL), the user has access, and the password is correct then only the server is left. Which is quickly crossed out by accessing the database through the server (which is running on the localhost). Now what? There’s really nothing else to change.

After banging my head against this all day (Well not actually all day. When I arrived this morning the power was out and so I had no access to the Linux box. When the power came on, there was no internet until a number of things were restarted. Most of the mess got sorted out by 11am.) I have come to the conclusion, with Paul’s help, that this Appache/PHP/MySQL bundle I’ve been using (which is called Xampp) may not be working correctly. There’s also the issue of MySQL being installed in two places on the machine (I didn’t know it was already there) so Word Press may be trying to access the wrong place. Or there’s the fact that .php files aren’t being recognized as PHP files unless I specifically run them by typing, for example: php install.php All this leads me to believe that I probably am going to have to uninstall Xampp and set up Apache, MySQL, and PHP the old fashioned way.

I better not have to re-re-re-install Linux.

Here’s a bit of weirdness: Last night I woke up at 3:43 in the morning. As I blearily stared at the clock I thought “3:43, that should be public static.”

Paul mentioned that he isn’t allowed to use most of what he knows in his college programming classes. If he writes tests, he has to strip them out. And if he were to use a Visitor pattern and his professors didn’t understand it he would be marked down. So he spends 9 months out of the year having to ignore what he learned in the other 3. That’s not cool. Speaking of the Visitor pattern, it looks like I’m going to get to take a class at Object Mentor in advanced object programming and patterns. David is scheduled to teach this class, but since he’s only taken it once, he wants to teach Paul and I first so that he can get his moves down and spot potential questions. Good for him and me because, although I’ve heard people talking about patterns in the office, I’ve only used the state machine pattern and not much else.

I wonder how test-friendly colleges have gotten in the past 5 years and if you make students write tests will they learn to hate them? I know a lot of students would ask me to teach them something about Quantum Physics because they had read a cool article or seen a ‘Nova.’ However, when they actually had to study about quarks and weak forces they were decidedly less enthusiastic.

So which one of the potential problems I outlined with the 5 minute install do you think is really the cause of its failure? The 2 databases? Xamp? The .php files not being recognized? An unsuccessful install of Linux? And how many more days before the 5 minute install draws to a close?

“public static” – funny.

My Apprenticeship - Tuesday, July 13, 2004

Tue, 13 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

What was supposed to take a half hour yesterday actually did take 30 minutes today (after we got yesterday sorted out). Email problems turned out to be mostly a matter of restarting Outlook a few times. With the SMCCSharp compiler taken care of, I could go back to my other project: getting Word Press, MySQL, and PHP running on the Linux box.

Although Micah and I had downloaded, installed, and successfully used Mozilla (a web browser – like Internet Explorer) I couldn’t find it today. Paul came over and he couldn’t find it. And he’s been using Linux for years! It was somewhere on the hard drive, but after wasting about an hour looking for it I decided to stop throwing good time after bad and just re-install Linux (for a third time) and this time make sure I installed almost everything. While that was chugging away, I used my laptop to look up Word Press, MySQL, and PHP. Word Press claims you can set it up in five minutes. The hidden assumption is that you know how to use MySQL and PHP. Here’s what I knew about MySQL as of this morning: It’s a database. Here’s what I know about PHP: it’s a scripting language. I did, despite my ignorance, manage to get an local Apache server up and running using MySQL and PHP. I even got to look at some sample databases using the local host (what all this means is that I set up the computer to be a server and then I had another part of that same computer start talking to that server using Mozilla – kinda weird but real useful when you want to get all the bugs out prior to opening your server up to the world wide web). After digging around on the internet for awhile I still wasn’t sure how to get a basic database up and running. I’m pretty hopeful about tomorrow because David found a book in the OM library entitled ‘MySQL and PHP for Web Development. Which sounds more than a little helpful.

Somewhere in the middle of the day, Paul said that he thought our SMCCSharp compiler might have some bad dependencies. Hmm.

I remember, during one of my first days at Object Mentor, being flabbergasted that David had a server running on his laptop. Weren’t servers huge things that hosted cnn.com? How could he possibly have one on that little laptop? I’m pretty sure I kinda didn’t believe him when he mentioned it in passing. Now here I was a few weeks later setting up a server my own self. Crazy talk.

4 days since distant-past-jake started the 5 minute WordPress install – how long will it take? Stay tuned.

My Apprenticeship - Monday, July 12, 2004

Mon, 12 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Today was a day of unintentional diversions. Paul and I came up with an idea of how to test an area of our state map compiler that had been eluding us. We would build up an input file, in the tests, and then send that input file into the SMC which would create an output file that we could read and check to make sure it works. If we had been designing this SMC thing from scratch using TDD, we could have made it easier to test, but since large parts of the program are a legacy, we had to resort to the building/writing/reading file method. But the parser in the SMC program doesn’t like to be called twice, so a fair amount of the day was spent figuring out how to get around that for multiple tests. The other part of the day was spent writing a bunch of tests that had to be thrown out because there’s a quirk of the program that given the same input, you may get different output. Everything will be there, but the order is not guaranteed. This is one of the problems with using a list, things may not come out in the order they were added. So our tests, which assumed a certain order, would randomly fail. Great. Luckily, there was only one test that really needed our new method of building/writing/reading so we could move all the old tests back to the old way (we called the program directly to set variables) and, in the one straggler, we re-wrote the test so that order was unimportant.

Pretty much the same thing was going on in the office. The email wasn’t functioning and everybody who knew about the mail server was either out of town or teaching a class. Then Quickbooks started malfunctioning and things typed in by one person were not available to someone else in the system Nobody was sure if data might be getting lost – this had the effect of chilling productivity in the office.

In short, everybody’s 9am plans never even got started.

“If we had been designing this SMC thing from scratch using TDD, we could have made it easier to test…” How many times over the last five years have I run into this exact same problem? Too many to count – lack of test leads to untestable code which is very often unmaintainable code.

Oh distant-past-jake, why didn’t you just loop over the list you were trying to test and see if the things you wanted were in there? Ah well, a testing technique I would soon learn the hard way

My Apprenticeship - Friday, July 9, 2004

Fri, 09 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Revision: the State Map Compiler C Sharp version isn’t done. It doesn’t actually work, as Micah showed us. I feel pretty bad about it because we should have tried the things he tried before showing it to him. The SMCC# produces code that’s meant to be part of a larger program. We really should have written the larger program to make sure it works. Bad apprentices. Bad!

On the positive side, I’m becoming a lot more familiar with how this SMC thing functions as we plow through it again (and again, and again). After looking at some UML with Micah we decided to re-design how the C# version of the SMC is written to avoid using inner classes. Which is good because they were confusing me.

David and Paul have suggested that they start writing an ‘Anti-Jake’ blog. Treachery! We shall speak of them no more.

Tonight I’m going to a ChAD meeting (Chicago area Agile Developers). It’s starting at 6:30 at the Loop-area DePaul campus, but my train doesn’t arrive until 6:25. So I’ve got 5 minutes to get off the train, unfold my bike, and peddle my ass on over to Jackson and Wabash.

Last night I bought my airplane tickets to XPAU (eXtreme Programming Agile Universe) which is being held in Calgary this year. I’m pretty excited about going ‘ should be a great chance to learn a bunch and maybe meet somebody who will offer me a job. (stop laughing – it could happen). The crazy part is that I have to be back to start my teaching gig the morning after the conference ends. At 8:30pm Calgary time, I get on a flight to Las Vegas, NV. Somewhere around midnight I get on another plane which takes me to Chicago. Arriving at 4:30am. Yipes! I guess I’ll sleep a few hours at the airport and take a cab to school. The alternative is taking the ‘El back to my home in Chicago, and then getting in my car almost immediately (gotta beat that traffic) to go back to school. That first day is gonna be a little bit crazy. Luckily, it’s only a day of meetings – no teaching required.

Yeah, you should probably run the code before you declare it done. “But all the tests pass” is not an excuse. In fact, it’s an insult to your test suite.

XPAU is now just called ‘Agile.’ I’m speaking at Agile 2009 in August in Chicago, it’s going to be weird going back there after all these years as a presenter.

My Apprenticeship - Thursday, July 8, 2004

Thu, 08 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Today Paul and I set up the conference room for next week’s class by making sure IntelliJ, FitNesse, and M.S. Visual Studio were installed and current on the mini computers. It’s gonna be a full house for the TDD (test driven design) class – six computers with 2 people at each.

Then Micah gave us a choice: one of us could try to track down a mysterious file on a computer about to be re-formatted and the other was to figure out Word Press and install it. I chose Word Press – it’s a program that can be installed on a server so that blogging (weB LOGging) is quick and easy. We are setting up a blog so we can start the site www.butunclebob.com where individuals can object to Uncle Bob’s (Bob Martin, founder of Object Mentor is often referred to as ‘Uncle Bob.’ I don’t know why either) frequent declarations (like: ‘Debuggers are bad’ or ‘Comments should not be used’ ) and then Bob can respond. But the servers are Linux based, so I needed to install Linux on an old computer they had lying around. Linux is an operating system( like Windows, or Mac OS) favored by nerds. Long have I wanted to get my hands on this O.S. ‘ it’s in all the great nerd literature like Crytomonicon or, uh’ hmm. Maybe something by Coupland? Anyway, the nerds favor this O.S. because, although it’s not the most user-friendly of file systems, it is very stable and secure. Linux servers don’t crash as often as windows servers but even when they do, it’s not too hard to get them back up and running.

So I installed Red Hat Linux, and then I installed it again because I did it wrong the first time. Yargh. After that I downloaded My SQL and PHP. These are programs that Word Press needs to run on a server. What they actually do, I’m not sure, but I’ll have to look into that later because, by that time, I had to go.

Wow, first day using linux. These days it’s easy to forget that I wasn’t alway in love with the *nix based operating systems, but there was a time when I was a windows developer. I remember getting placed on a Rails project and trying to get it up and running on my windows laptop – I think it took about a week. Eventually I gave up and dual booted Ubuntu. After that I made the switch to OSX.

I love the part where I explain that Linux is more reliable than Windows and what a ‘blog’ is. Who did I think was reading this? As if me explaining every 20th technical term was going to make this blog any easier to read. Actually I didn’t even think of those original posts as a blog – It was just sort of an extension of the little stories I’d been telling for years on my crappy website.

Also, take note of today’s date and keep track of how many days it takes me to install WordPress. At the time their website boasted a “5 minute install.” I might have gone a little over that estimate.

My Apprenticeship - Wednesday, July 7, 2004

Wed, 07 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Most of today was the State Map Compiler. We got it to work, but almost all the work was being done in a 600+ line class. Armed with a bunch of passing tests, we proceeded to brake just about all of them in the refactoring process. But, and this is important, we only broke one test at a time. The goal was to offload a whole mess of functionality into a bunch of other classes. By the time we got back to a refactored green, we had about 100 lines in the controlling method and much more readable code. Whoo!

Then it was time to do a real conversion of the OMwiki to FitNesse. Previously Paul and I had used a copy of the OMwiki files for practicing purposes. Today we shut down the OMwiki and ran our program on the wiki pages. Due to an unfortunate use of a backslash in a program designed to deal with Unix, our conversion program created a few thousand extra files. Sigh. You see, the Unix system uses forward slashes to denote directories so it sees backslashes as just another character. All those directories we thought were being created became strangely named files. And all the real files got dumped in the main directory. Too many files to delete, oddly enough. The ‘rm’ command stands for remove in Unix-speak. But apparently it has a limit as to how many files it can address. We actually had to write a program in Python just to delete the files. Then, on our second try we saw the resurgence of the strike-through (double dash) problem. Apparently there had been some changes after I left yesterday, but we went back to an earlier solution and that worked. As I had to dash out the door to make my train, everything was working on the new OMwiki site except for an unfortunate spelling of refactor (as ‘refacator’). I’m sure they’ll be able to fix that one without me.

Anyway, I’m putting today in the win column.

I’m not really sure why we couldn’t have just copied the files, ran the program, got it up and working on the new wiki, and then shut down the old wiki. That would have involved too little stress I suppose.

As to the State Map (Machine?) Compiler, I think that may have been the first time I was able to see the power of having a comprehensive test suite so you can refactor at will. More and more I work with developers who’ve had testing forced on them and see it as a necessary, or perhaps unnecessary, burden. I really don’t get that – I find a well tested application very satisfying.

My Apprenticeship - Tuesday, July 6, 2004

Tue, 06 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Fun things you learn when Micah returns: Your tests? – don’t really test what they should. The regex expression you labored over? – can be done a lot better. All the time it takes to run your program? – is because your program is doing 20 more operations per file than it needs to. All that polymorphism stuff you vaguely grasp? – you don’t know at all.

All part of the learning process.

Other than that it was a pretty good day. I got back from Vegas last night and was pretty tired (Cirque de Soile’s ‘O’ and ‘Mystere’ are amazing. Stop reading this blog and go see them now). Before Micah knocked some sense into our heads, Paul and I spent most of the day working on the State Map Compiler. Writing tests for this thing sucks. The output is basically one long string (which is what a program is) so the tests are mostly comparing lots of strings. All the excitement of: missed spaces, absent newline characters (‘\n’), and improper capitalization.

A recurring theme in any apprenticeship program: The mentor checks in with the apprentices and points out all their mistakes. Since I was writing this blog in hopes of getting a job I edited out a lot of these revelations so I didn’t look like quite such a dumb-ass but, in truth, this probably happened every few days (if not daily).

Fun side note: Laura (my wife) and I went to Vegas that time because she had successfully completed a sticker program. After having our millionth conversation about how she needs to clean up her study, car, office, etc she suggested a sticker program (You know the thing where every week, if you complete the task, you get a sticker in a cell on a table?). I thought she was kidding, but it turns out she wasn’t. So we created the “Super Fun Sticker Program for Increased Cleanliness” which lasted 6 weeks if I remember correctly. If she got all the stickers then we went to Vegas and saw some shows. After I tell this story people either tend to think I’m a terrible husband for subjecting her to some sort of behavioral experiment or a very nice husband for encouraging her quirkiness.

My Apprenticeship - Friday, July 2, 2004

Fri, 02 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

We did use my code in the wiki conversion program. And while it was good to finish up that project, this meant we had to start working on the State Map Compiler. A program first written in 1993 in C++, then later re-written in Java in ‘98. No tests, very long methods, and the documentation gives lots of examples in C++ (which I don’t know much about – it seems like Java, but it most definitely is not.) The point of the program is to take in a simple statement of a ‘state machine’ and output source code that implements the state machine in either Java or C++. We are going to be adding a C# output option to this program.

Now, at this point, you might be wondering: ‘What is a state machine?’ Well, let me attempt to regurgitate the turnstile example. A turnstile has two states: Locked and Unlocked. If you try to pass through a turnstile when it’s locked, you’ll get quite a different response than when it is unlocked. A coin will cause a locked turnstile to become unlocked, but won’t do much to an unlocked turnstile. A lot of software engineering problems can be thought of a nothing more than a dressed up turnstile with a few more states. So it’s handy to have a program that takes in some information about a finite state machine and then outputs the basic code that you can build your implementation around.

Also, it’s pretty interesting to be working on a compiler. Software that writes other software (that, ultimately, will be translated into byte code, and then the byte code will be translated by the virtual machine so that the real machine will know what to do – quite a process).

Not a bad explanation of a state machine for guy who just learned about them that day: Good job distant-past-jake. I think ended up using a state machine pattern in my code submission to ThoughtWorks later that year.

I later worked on a huge java project that would generate something like 15 classes for every one written class. I soon got very tired of generated code (“Hey, why did my changes go away? Oh, it’s a generated class… (next line said forlornly) Guess I better go dig through the xml pit for what to change.”)

My Apprenticeship - Thursday, July 1, 2004

Thu, 01 Jul 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

There’s just one problem with our conversion program: the Double Dash (cue the dramatic music). In FitNesse, two dashes in a row (–) mean strikethrough. But in the old OMwiki, two dashes had no meaning. A fair amount of people use two dashes to separate their thoughts ‘ as you might expect. So our pages were getting translated with lots of text ‘struck out.’ Okay, so all we had to do was write a program that looks through all the wiki pages and replaces two dashes with one. Which, it turns out, is not as easy as we thought. The proper way to do this is with regular expressions (mentioned in the previous post), but neither Paul nor I understand them too well. So we looked around for the regex (regular expression) book at OM, but somebody had probably taken it home for some light reading. With that option crossed out we started to build a series of nested if statements. After getting about 5 if’s in, Paul turned to me and said ‘If Micah sees this code, we’re telling him that YOU wrote it.’ Fair enough. Well, the problem with nested ifs, as you might know, is that it’s hard to keep all those conditionals straight (if this, but not that, and this, that, and the other, wait! No, not the other. Or, uh, let’s start again.) so we quickly got lost. Then we gave up. It was just too messy to keep track of how long the string of dashes were. We wanted to change two dashes in a row, but no other length was to be affected. So we started looking up regexs on the web. An hour or so later, it was time to go.

Incidentally, I made my train with only a minute to spare. Really, really, big wind today.

Here’s the weird part: I just figured it out while on the train. Using Java’s API (a guide to all the classes and methods in the Java library) and some tutorials I downloaded off the web, I talked the ‘matcher’ into finding dash strings of any length. If the length of that string was two, I replaced it with one dash. If not, I kept the string the same. Nice.

Gotta say, I feel pretty good about being able to contribute something to this project.

So mark down July 1st, 2004 as the day I felt like I really might be able to become a programmer. Since starting at Object Mentor I had contributed almost exactly nothing except for the occasional “You spelled that wrong.” But on that hallowed day I actually figured something out before Paul. Maybe I wouldn’t have to go back to teaching high school remedial physics and hating Sunday nights because of what Monday promised. Heady thoughts.

Interestingly enough Jay Fields wrote a blog post about the “Spellchecking Pair” a few years later. He makes some good points about matching the levels of programmers but:
A. That’s not always possible.
B. I’d feel pretty bad if I was a junior guy on his team when he wrote that post.
C. Inexperienced devs already feel like they are worthless – try not to make them feel worse.

Still it’s important to talk about the problem – you want to bring developers along but sometimes the impedance mis-match can really frustrate both developers. I generally err on the side of “pairing increases the long term health of the project,” but of course their are exceptions. It’s just that the exceptions have a nasty way of becoming the rule.

My Apprenticeship - Wednesday, June 30, 2004

Wed, 30 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Things got much better today. Much to my embarrassment, I drove the keyboard for a few hours. I’m slow, I can’t type very well, I don’t know IntelliJ (the slick Integrated Development Environment (IDE) that Object Mentor uses for writing Java code), but I banged out a few tests and made ‘em pass. The problem we were having is that you can attach files to wiki pages in the Old OMwiki. But in the new, FitNesse-centric, wiki we want to put all the files in one area and have links on the pages. This is mostly just a matter of changing the file structure, but it does have the added advantage of being able to link to files from more than one wiki page (well, you could probably do that in the old wiki, but solution would be inelegant). So we wrote a program that checks the folder (of the wiki page) for files and then creates a link to them at the bottom of the page. And it worked.

Mostly.

Some of the files had spaces in them which made our wiki widgets quit while only part of the way through the file name. But, in the plus column, we were able to completely transfer over all the wiki pages and have them look pretty much the same. Right now Paul and I are working on a program to grab all the filenames with spaces in them and cut out the spaces.

The eighties music was off today (which is good because I was getting a little tired of ‘Tainted Love’) , and Paul played some Beatles on his desktop-sized laptop.

“The eighties music was off today” – huh? I don’t remember this at all. Were the Object Mentors crazed 80s fanatics?

I’m still scared about pairing with people I don’t know, oddly enough. I’ve been doing this pairing thing for 5 years now and yet I still get wound up at the prospect of sharing a computer with another programmer – what if they’re better than me and I feel bad? Almost 100% of the time my fear is completely unfounded and I end up having a great time and learning a bunch. The only time I really hated it is when I let the other person type too much – then I just feel like a loser. No matter what the skill difference, each pair should spend about the same amount of time driving.

My Apprenticeship - Tuesday, June 29, 2004

Tue, 29 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Well, I didn’t drive today, but I did make some progress toward figuring out what was going on. We built a few custom widgets that would translate wiki text that was problematic. For instance: Putting three single quotes around something

’'’something’’’

makes it appear in bold, and two are italics, so 5 should be both italics and bold, right? Not exactly, the FitNesse wiki was only catching the first 3 single quotes. So we used a regular expression (what’s a regular expression? A bunch of symbols, which kinda look like swearing

”’’’’’(.+?)’’’’’” or ^#[^\r\n]*(?:(?:\r\n)|\n|\r)?

that look for patterns: like a word or words surrounded by five single quotes) to build a custom widget to catch just such a case.

Susan’s computer started making this squeaking noise about a week ago. Now it makes a pretty constant background variable squeak/whine. And the parts aren’t due in until next week (It’s the fan on the processor that’s making the noise, so we’re a little hesitant to douse it with WD40). Paul sez ignoring it is an exercise in concentration, but I read somewhere that periodic high pitched noises are the hardest to ignore (easiest are low pitch, constant noises).

“I didn’t drive today” – read that as I spent all day watching someone type. Which is not a fun day. Chalk that up to the fact that we were doing regular expressions and I had never seen one before. I’m decent at RegEx’s now but I think I avoided them for years because of my first experience. Nothing like resolving to get more involved and then falling out of the frying pan and into the fire to dampen the old spirits. And then a whining computer in the background for extra torment. I’m surprised I didn’t just type a single four letter word to sum up June 29th 2004.

My Apprenticeship - Monday, June 28, 2004

Mon, 28 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

In the morning I installed 40 meg or so of windows ‘critical security updates’ and after lunch I installed MS Visual Studio. Then I went home.

But seriously folks, that was a fair chunk of my day (2.5 hours alone to install MSVS). Paul and I paired on the OMwiki conversion (to FitNesse) project. I can’t say that I’m understanding much. We had this big problem were we couldn’t use the ‘get’ and ‘set’ methods, but I’m still not sure how we solved it. We did manage to get the conversion program to move all the files to the FitNesse wiki and emulate the FitNesse file structure. But we didn’t actually convert the individual pages to FitNesse wiki text yet. I’m so out of it that, when I saw the program run and the files appearing, I thought somehow Paul had finished the conversion right under my nose. I’ve really got to stop him more often. He understands soooo much more than me I just can’t keep up. Micah made a large scale drawing of the program on the whiteboard, but I’m having trouble applying that UML diagram (which I mostly understand, despite my lack of UML training) to the code we are producing. Tomorrow I need to look stupider and feel better about it. This staying quiet and trying guess what’s going on isn’t working so well.

The funny thing is that, as a teacher, I know that students must take an active role in learning. Which is quite hard if they’re way behind. I’ve seen it before – when a student gets lost in a class (maybe they missed a few weeks or just took a mental vacation) they tend to shut down instead of asking a lot of stupid questions. Maybe I should drive the keyboard for a while. That ought to be seriously embarrassing.

This is one of about 10 posts where I say something like: “What the hell happened today – I need to ask more questions.” Part of my role at Obtiva is to say this to our apprentices: “Yes I know you feel like a complete idiot and it seems like you can’t take one more day of not knowing what the hell is going on, but believe me that I’ve been there and it will come. Don’t be afraid to look stupid early instead of confessing at the end of a week/month/year that you have no idea.” Easy to say, but hard to do.

Also the computer I was using was this Dell Inspiron laptop which was about 2 years old at the time and very slow. Installing Microsoft Visual Studio nearly killed it. I think I actually still have it in the other room - I keep meaning to throw it out but it’s hard to let go of tech stuff.

My Apprenticeship - Friday, June 25, 2004

Fri, 25 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

We finished the C# class today with an exercise devoted to Threads. In case you have some operation that will take lots of time (like, say, writing to a file) you can send it off on its own while the rest of the program continues on its own way. This is called MultiThreading and it’s pretty cool. We designed a thread that scans a directory (folder) every second to see if any text files have been placed in it. If it finds any, it moves them to another folder. We forgot to kill the thread when we were done with it, so the test kept running after we thought it was done. When we tried to manually place some files in the forbidden directory they automatically moved away. Because the cowboys and I are nerds, we created a whole bunch of text files and tired to jam them in the folder before the program could catch up. Alas, the program was too fast for us.

Micah’s gonna be gone next week so the apprentices will be on their own.

The really weird thing about that week was that, up until that moment in my life, I truly sucked at puzzles. Somebody would start in with “You and three friends are walking through the forest and each of you has a prime number of hotdogs…” and I’d get lost and start to panic. I’d be thinking that everyone knows the answer to this but me and they are finally going to find out that I’m a fraud and the dumbest guy in the room. So I’d spend most of my thinking time worrying about how much time was left before someone figured it out and then what if they announced that they had got it but didn’t tell me the answer and I had to keep thinking about it knowing that someone had beat me to it and it was probably obvious and… Etc., etc., etc.

The weird thing about that week was that I solved every puzzle without getting crazy. It was like after I got the first one I could relax and think about the problem. It’s a constant surprise to me how many things essentially boil down to confidence and comfort. If people feel comfortable in a situation they will perform exponentially better than they will in a tense environment.

My Apprenticeship - Thursday, June 24, 2004

Thu, 24 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

You Cowboy has taken to saying ‘sankyou’ when Cowboy helps him out. Cowboy replies with an even more Asianized ‘sankyou’ and we move on. You Cowboy is also fond of saying ‘RefRactor’ and ‘Ohh Cowboy’’ (with this sort of world-weary sigh that’s just perfect) There were times when I couldn’t code because I was laughing so hard. A quality experience.

The pattern that established itself is this: Micah introduces some concept in C#, such as event handling, and we stare at him confusedly. Then we do a project involving said concept. I get really confused, while the Cowboys trade barbs. Eventually we figure it out. And Cowboy checks his eTrade account. Repeat cycle.

Incidentally, ‘delegates’ are really cool. They are methods that you can pass around. I have this problem in my Tic Tac Toe game where I keep having to cycle through all the spaces in the board. So my code is littered with things like this:

for( int row = 0; row < 3 ; row++) { for(int column = 0; column < 3 ; column++) { [[some function or other]] } }

The functions in the middle do all sorts of different things. I think that with a delegate I might be able to have just one place where that nested ‘for’ loop happens, but with an assignable method inside. Which would be cool. Except that TTT is written in Java and delegates are in C#. Micah thinks that using ‘runable’ will work, but David thinks it’ll be tricky (David thinks it’d be a snap to do in Python – always with the Python). Anyway, I’m gonna try.

When pairing goes well it’s like hanging out with your friends all day, getting work done, and getting paid to do it.

To be totally honest I never did try to use Python or ‘runable’ to solve the TicTacToe repeated code problem. I did later learn Ruby and blocks which solved the problem quite nicely. As my friend Pat Farley once said: “Ruby is great at removing the duplication where you’re doing the exact same thing repeatedly but all the words are different.” I had to think about that one for a week before I got it.

My Apprenticeship - Wednesday, June 23, 2004

Wed, 23 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

More fun today with ‘Cowboy’ and ‘You Cowboy.’

We got to a point where I was totally lost. Finally. Monday and Tuesday were all about picking up small things and differences from Java. Today we got into some full on Polymorphism and it was cool/confusing. Cowboy and You Cowboy were all over it and, unfortunately, I was driving (had control of the keyboard). Cowboy started talking a mile a minute while You Cowboy had some helpful suggestions about topics unrelated. I had just about no idea what I was typing. I don’t think pair programming works very well with ‘pairs’ of three – sometimes there is too much going on and rational thought is impeded. Later I was able to catch on and here’s what was happening: The CheckBook can take in Deposits and Checks, both of which are children of the Transaction class. Today’s big task (amongst others) was to give the CheckBook class the ability to step through all the transactions and reorder them. That meant the CheckBook had to inherit from the Enumerator class and the Check class needed to inherit from Enumeratable. And there was much Overriding and passing of classes and the casting – don’t forget the casting. In fact, just thinking about it makes me want to look over the code again. I’ve got to get M.S. Visual Studio installed on my laptop. Instead, I’ve been having fun using OM’s super-fast mini-computers. They have these cute little Dells with flat panel screens that can fit four to a box in shipping crates. Pretty handy if you need to haul a bunch of computers to a conference (like, say, the XP/Agile Universe conference August 15-18 in Calgary! Whoo!).

I got another puzzle right today. What’s up with that? If you want, here it is: You are in a room with three switches. In another room, not visible from your room, are three lights. Each light is controlled by one switch. All the switches are off now. If you leave the room to check on the lights, you can’t come back (the door locks behind you, poison gas steams into the room, dogs are released that shoot killer bees out of their mouths when they bark, etc.). You can play with the switches as long as you like but when you go to the other room you want to be able to know which switch controls which light.

Remember: Penn and Teller say that people who love puzzles must die.

Oh ‘Cowboy’ and ‘You Cowboy!’, your antics still amuse after all these years. Not so pleasant memories about casting classes into other classes. That whole statically typed thing seems like such a fad in hindsight. I shouldn’t really trash talk about Java and C# though. I think half the reason I react with aversion to them is that I was a mediocre programmer back when I was in static land. When I was in Ruby I really came into my own as a programmer and so I associate Java and C# with the times in my life when I sucked and Ruby with some actual competency and satisfaction.

Also, I love the part where I was impressed with the flat panel monitors – this was 2004 when only the serious would shell out tall green for a 15 inch monitor with thousands of colors and a 1:50 contrast ratio.

My Apprenticeship - Tuesday, June 22, 2004

Tue, 22 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

More C# today. I was working with two paying customers: Tony and Jeff. Watching those two bounce off each other was a riot. Jeff kept wanting Tony to type faster and Tony got even with Jeff by adopting an Asian accent (Jeff’s Asian). Jeff wasn’t super impressed with either TDD or refactoring so he kept advocating shortcuts, but Tony kept telling him to slow down ‘Cowboy.’ Why ‘cowboy,’ I don’t know but it was pretty funny.

We wrote a checkbook program today and we got to use polymorphism. A checkbook can take in deposits or checks, but both are really transactions. So we wrote an abstract Transaction class and then had the Check and Deposit classes inherit from it. It passed the tests, but I need to look it over because by the end of the day we had given in to Jeff’s demands and started just doing what he said. Things went kinda fast after that.

In the category of strange but true: I solved not one but two puzzles today. Normally I end up visualizing the person who offers the puzzle as a more gasoline soaked combustible version of themselves. But today things were different.

You should stop reading now if you don’t like puzzles. You have two lengths of rope that will each take 60 seconds to burn. The rate of burning, however, is not constant. So one rope may burn through 90% of it’s length in 5 seconds and take 55 seconds to consume the other 5%. The ropes are not identical. While one may do 90% in 5 seconds, the other may do the first 90% in 40 seconds. The only thing that is sure is that each will take 60 seconds to burn all the way. You have a box of matches, but no access to a timer. Your goal is to use the ropes to measure 15 seconds.

Fun fact: The two guys worked at a place in my old neighborhood where I got arrested and thrown into jail. Lemme give you some advice about getting arrested – always carry 100 dollars on you in cash so you can bail yourself out. Most non-terrible offenses have a thousand dollar bail, but they’ll accept 10%. The ‘jail’ was a bit of a let-down as it was really just a room with a locked door and a pay phone. On the plus side I got to call my Dad from ‘jail.’ Remember that old ‘Was not was’ song ‘Hello dad, I’m in Jail’? Well I do.

My Apprenticeship - Monday, June 21, 2004

Mon, 21 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Long bike ride in the rain this morning. Only got scary when those 16 wheelers zoomed by throwing a few gallons of water at me and my wiener bike. Micah was teaching the C# class today and lots of it was familiar b/c Java and C# are pretty darn similar, but it was cool to review TDD best practices while working with someone who didn’t know about ‘em. I was paired up with one of the paying customers (‘Tony’) and we managed to figure out M.S. Visual Studio. It’s kinda like FrontPage in that it tries to force you to do things the Microsoft way. The refactoring tools are pretty crappy – no extract method option (which is my new favorite thing)! So Tony and I had to do our own refactoring which was a good exercise. C# does have a cool decimal class which gives you arbitrary precision up to 26 places. This is very cool for money programs, because, as we all know, that guy (Richard Pryor) in that Superman movie used the rounding errors to steal money from banks and we can’t have that. Anyway, I feel like C# isn’t going to be to hard a transition, but it should be said that today was only the beginning.

I was sleepy all day today because we had this late-night cast party at my pad this weekend. Laura’s cast came over and brought their friends. All in all it was about 25 people. I’m glad my landlord (who lives below me) wasn’t home because there was lots of loud music and dancing. Right in the middle of Micah’s lecture he looked over at me and caught me in this huge yawn – for the record it was the sleep deprivation and not the class.

In Tic Tac Toe news: My program is now passing 11 out of 11 tests! And the methods tested include minimax and makeTheComputerMove. Now all I gotta do is figure out how to test the human interface. Hmmm.

Yeah that bike ride was kind of a harrowing adventure each morning. The only road that went directly to Object Mentor was a four laner that got lots of traffic. Could be pretty scary.

My Apprenticeship - Friday, June 18, 2004

Fri, 18 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

I arrived this morning to find that the Martins are back in town. Bob seems like a good guy. I didn’t talk to him much, but working next to someone all day can give you a good impression of a person and Bob seemed easygoing and fun. He also was trying to quote some obscure line from a classic Star Trek episode (you know the one where the crew of the Enterprise have been captured by an alien being (again) who wants to learn about humanity (as always) by having them play out some bit of 19th or 20th century history (this time it was the shootout at the OK coral)? Well, I do.) so that gets put in the plus column. Micah took a look at our project. Then he asked us what we thought the project was.

Uh oh.

‘Er, to change the OMwiki python code so that it operates more like the FitNess wiki?’ Well, not really, as it turns out. What he really wanted was for us to translate the OMwiki pages into FitNess wiki pages so they could kill off the old OMwiki and have it be re-born with all the added features of the FitNess wiki. ‘Oh.’ We said.

So we start over.

One very cool thing about today is that Micah offered to let me sit in on the C# course that he’s teaching next week. I was worried that I might slow the class down because I’m an absolute beginner (I know it’s a Java knock-off, and that’s about it), but that’s who the class is for. Excellent.

Paul and I started writing the wiki conversion program – TDD style of course. There will be no coding without a failing test. After a few days of Python’s super easy file writing techniques, we had to stumble around with Java’s roll your own types. Basically, Python is a higher level language than Java so it does a lot more for you. Yes, I grudgingly admit that Python has advantages over Java – There, I’ve said it.

When I first started re-reading these blogs post I thought to myself: “Funny, I don’t remember that we spent a lot of time fixing up the old Python wiki…” Yeah, that’s because we didn’t – We burned a week working on the wrong project. I felt so stupid. However, it’s an important point that when XPer’s start going on and on about having an onsite (or highly available) customer you should probably listen. I’ve seen this situation many times since: Only person who knows what they want is totally unavailable and so the developers work on their own only to realize that they totally misunderstood the requirements. The awesome thing is that Object Mentor makes its living teaching people how to be Agile/XP and they still made the classic off-site customer mistake. This isn’t to bash on them, but merely to say that when the major stakeholders are unavailable (or perceived to be so) then you’re going to waste a lot of development time.

Also, Uncle Bob is a nerd’s nerd. One time someone was using an old-school dial up modem in the office while he was around. Just from listening to the “scree scraa scraa” noises it made, which he casually heard from across the room while working on something else, he diagnosed some problem or other.

My Apprenticeship - Thursday, June 17, 2004

Thu, 17 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

You know what? I had fun today. Imagine that. Yes, some of the day was spent looking at the OMwiki pyhton code which still confuses the hell out of me, but I learned how to extract methods in Eclipse. I know I’m not the first person to say this but, the ‘extract method’ thingy is way cool. It makes it sooo easy to slice large pieces of crazy code into much smaller, friendly, bits. Almost fully automated creation of methods. Neato.

The other fun thing I did today was getting my Python Tic Tac Toe to not only run, but to play perfect. It takes forever, though. David and Paul say it shouldn’t take that much longer than Java, but it does. If the TTT board is empty, it takes about a minute to make a move.

Yet more fun, I’m doing a TDD (Test Driven Design) re-write of Tic Tac Toe in Java. I’ve hidden the original code away and I’m writing the tests first using the Junit plug-in (this program works with Eclipse to make running tests easy). When your test passes, you get a green bar. Failure is red. It’s amazing how addicted you can get to seeing that green bar.

Humbling moment of the day: I’m eating lunch with Paul and I’m describing my Tic Tac Toe program. Somewhere in the conversation he casually mentions that He, Micah, and Chris (former OM employee) each wrote their own chess programs awhile ago. For fun.

Yipes!

If you don’t play chess, lemme tell you: It’s scary complex. It took Paul six months. Paul’s actually been working for OM for about 4 years now. He spent the first 1.5 years working year round when not at school. But then he settled into just working when he’s off school. Paul’s thinking of getting a Master’s in Math.
No Micah today, but he should be in tomorrow. Paul has made some good progress on the OMwiki widgets so I feel much better about the project than I did Monday or Tuesday.

Writing a chess program for fun – still blows my mind. I believe they had their chess programs play each other. I aspire to someday be that geeky.

There was a time I thought I would never go back to a non-refactoring IDE, but I’ve been using TextMate for a few years now. Although, one of the guys at work uses RubyMine and I have to say that it’s coming along quite nicely. I’ve had my heart broken with Ruby IDEs so many times in the past that I don’t know if I’ll ever love again.

My Apprenticeship - Wednesday, June 16, 2004

Wed, 16 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

The first part of the day was more crazy python code. I was asking Paul some more questions about the Visitor pattern and at some point, while he was describing some aspect or another, he said ‘It’s kinda like a linked-list.’

This happens a lot. Somebody at OM explains some complicated thing with another complicated thing and I have to take a deep breath and say ‘So, what is a linked-list?’ At which point paper comes out and drawings are made and I realize that I hadn’t really understood arrays until then. I’m so behind the curve in terminology it can be overwhelming. But, I should add, this is a thousand times better than being bored in my Java class. From way too slow to way too fast in one month.

Total immersion is supposed to be a good way to learn a language. I hope so.

We (and when I say we, I mean 95% Paul) did make some good progress on the OMwiki widgets. We’ve got the widgets successfully passing 5 or so tests. But the wikiWord widget is giving us problems for reasons I don’t understand. Something to do with a class/method we can’t call from where we are.

I, however, finally got my Tic Tac Toe to program to run without giving me some crazy sort of error. Whoo! That success was tempered by the realization that the program now plays very poorly. Wha happen? One of the big problems turned out to be Pythons’ way of sending the object as an argument (the infamous ‘self’) We ran into that problem a lot in our OMwiki project. Many methods have to take in self and then, to address one of the object’s variables, you have to write self.myVariable instead just saying myVariable.

On the way to the train today my bike’s gear shifter wire slipped out and the bike defaulted to 3rd gear. Hills became interesting after that. The good people at Rapid Transit (Excellent bike shop) fixed it in 5 minutes for free! Everybody reading this (all 5 of you) should go buy something from them right now (corner of Wolcott and North in Chicago). Later, Bryn and Tina came over to my pad and we indulged in some drunken video game funness (in the interests of full disclosure, we played: ‘The Simpsons: Hit and Run.’ You know that one level, where you have to use the Globex Supervillain car to demolish all the free laser gun stands, and then get back to KrustyLu studios before time runs out? That’s the level we couldn’t beat.)

“The Simpsons: Hit and Run” is/was one of those truly great games I still think about from time to time. I wonder if I still have it… Damn! I must have sold it when I got rid of my original Xbox to that guy I met though Craigslist – he was buying the Xbox and most of my games for his 7 year old son. Only later did it occur to me that I had a number of Grand Theft Auto titles mixed into that bundle. Oh well, that kid’s gotta learn about graphic violence someday.

The ‘Linked-list’ moment still stands out in my brain because just before telling me “It’s kinda like a linked-list,” Paul had this great look on his face like he had finally thought of a super easy way to explain this concept. He was so clearly pleased with his analogy that I felt just awful fessing up that I had no idea what he was talking about. The weird thing was that, as a teacher, I was totally used to this situation – just on the opposite side of the learning curve. But that was my job that summer: To get my mental ass kicked every day, learn from it, and come back for more.

My Apprenticeship - Tueday, June 15, 2004

Tue, 15 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Rough day.

There’s only so long I like to stare at a mountain of code which doesn’t work and I don’t understand. Paul was frustrated b/c we still couldn’t figure out why the tests of the OMwiki were failing. We sent Micah an email and his reply was pretty much, ‘Oh yeah, I’ve run into that problem before. I don’t remember the solution but it’s one tiny thing that causes the whole suite to fail.’ Nice. Well, Micah’s on vacation so that’s fine, but it’s maddening to know that one line of code may be preventing you from even getting started. Welcome to programming.

We spent the morning banging our heads against this wall, then, having accomplished just about nothing, we went to lunch. Paul thinks John Kerry is crafty for being spineless. I think he’s just spineless. We both think he stands a good chance of being elected.

Later we stared at OMwiki some more. It’s kinda depressing but I still don’t have a real good grasp of ‘this’(in Java) or ‘self’(in Python). I asked Paul for an explanation but I only sorta understood the answer. Paul sez we’re trying to use a ‘Visitor’ pattern on the OMwiki code, like is used in the Fitness wiki, but I only have a tentative grasp of the ‘Visitor’ pattern. It has something to do with avoiding a big conditional method by creating a lot of smaller methods ‘ each of which will translate some different bit of wikiLanguage into the appropriate Html. The cool thing about wiki pages is that you type in certain commands and the code (which we are working on) automatically sets up links, or makes things bold, or does other cool formatting things. At some point, Paul figured out that the vcs (version control system) check-in function was making the files read-only (if you recall, that was our big problem from the beginning). So he commented it out and the tests all passed. Whoo. We’ll need to put it back in as soon as we know how to do so (and not bring it all down around our ears), but for now we can work on the wikiWidget (the module that translates user text into links and stuff) and actually be able to test it. During a free moment, I download TextPad and the Python plug-in that lets it color code python script. Good program ‘ way better than Word Pad. Unfortunately, my Tic Tac Toe program will compile but it won’t run. Actually, for a time it wouldn’t do anything. I’d type in:

python TicTacToe.py

(from the command line) and … nothing. No syntax errors, but no game either. I had to take the ‘main’ method out and put it on its own. Now, of course, there’s all sorts of run-time errors. I keep trying to call methods with no parameters like:

print aBoard.toSting()

and Python keeps telling me I passed it 1 argument. Hmm. I’m working on that one. I’m also having trouble with my program seeing global variables such as the TicTacToe board (myBoard). I kinda wish I could spend some more time working on Tic Tac Toe with David before I look at all that OMwiki stuff in Pyhton. I’m not much use if I can’t even execute a simple toString call.

One last thing: Just to complete my day, I missed the train home by zero seconds. Zero, you say? Yep. I just made it but the asshole Metra guy said ‘no bikes’ and let the door shut in my face. For the record: the bike folds up and goes in a bag (which is perfectly Metra-legal). So I sat around for another 62 minutes waiting for the next train.

Get ‘um next time, tiger.

I can still remember missing that train. Some days there was wicked wind coming in from the lake and it would add about 10 minutes to my bike ride – this was the first time I discovered that. I was peddling my ass off to make that departure and checking my watch every 5 minutes. And then to make it just in time only to be shut down by some jerk. Ah, memories. I’m pretty sure I shouted obscenities for a full minute.

As to my problems with Python – It’s funny to see me talking about how I don’t understand Python’s ‘self’ and then wondering what’s this weird extra parameter that’s getting passed along to all my methods. If you’ve never programmed in Python you have to declare ‘self’ as the first parameter of all methods. Which can be a little confusing to those new to the language.

I love, btw, that our big solution to the problem of the tests that wouldn’t pass was to comment them out – not very XP. We did spend a bunch of time trying though, so I guess that counts for something.

My Apprenticeship - Monday, June 14, 2004

Mon, 14 Jun 2004 00:00:00 +0000

In the summer of 2009 I revisited my 2004 summer apprenticeship at Object Mentor. What follows is the original 2004 post and then some 2009 commentary.

Today I took the train to Waukegan, and then unfolded my bike and rode the last five miles to OM.

Good news:

An hour to do some work on the train.

10 miles per day should help me lose some weight.

Bad news:

My commute now takes about 2 hours.

There’s a lot of hills.

Now that I’m using Python, I tend to indent everything.

Speaking of Python, I spent some time this weekend translating my Tic Tac Toe program into python. Most of the stuff is pretty easy. But it still took a fair part of today to complete the translation. And when I say complete, I mean it compiles, not that it runs. I’m having trouble printing out the board (my toString method won’t cooperate) and I haven’t quite got the hang of making complicated calls in python.

This morning I met Paul, the other apprentice. He’s been working with OM for about 3? years now. He’s finishing up a Computer Science degree at DePaul. During lunch I asked him what he though about apprenticeship vs. college and he had this to say: ‘Computer Science’ what a joke of a degree.’ Hmm. How so? ‘I learned more from watching Micah program in a day than I learned in my first year.’ Strong words. When I pressed him as to why he feels that academia can’t teach CS, he had two points: One, CS is so new and evolving that how to teach it is continually way behind the curve. Two, it just doesn’t work unless you have someone right there to help you. As a teacher, I can agree with the second statement. One of the reasons home schooling routinely kicks public school’s ass is because you just can’t compete with the Teacher/Student Ratio. I’m a way better teacher than just about every home-schooler, but who cares when I’m divided by 30 or even 20.

For awhile Paul and I did some pair programming (my first pair programming experience, aww), sorta. He’s way farther along than me so I was only able to contribute a few things, but even so I feel reasonably good that I could do something besides sit there. Unfortunately, we ran into a pretty sticky problem just about right away. While looking at the OM wiki code(written in Python) we found that none of the tests would pass. This is a huge problem because the whole TDD thing requires that you keep checking if the code works. Well it doesn’t. Except it does: The Object Mentor wiki is alive and well as we speak. So something’s wrong with the tests. It seems that the tests create some temporary files when they start and then delete them after a given test is run. Problem is that the files it creates are read-only, so it can’t delete them. We spent a few hours trying to figure out how this could be so, but we were flummoxed. Unfortunately nobody was around who had ever worked with the code (David just started, Lowell is on the business side of things, and others were involved in teaching a class). So we spun our wheels for awhile. At first it was kinda cool asking Paul what various code was for, but after a time he and I were getting more and more frustrated looking at code that really should work. And when you can’t test, you can’t do TDD. Eventually Paul started coding ‘blind’ – as he put it. But by then the day was over and I had to get back on my bike. Man my ass hurts.

This was the first of 3 different summers that I rode my bike excessively to work. Every summer I was convinced the extra exercise would help me lose weight and yet it never did. I guess I ate exactly the right amount of extra calories to keep the weight on. Awesome.

Paul, in the above post, is Paul Pagel. He later went on to co-found 8th Light with Micah Martin. We spent a lot of time that summer pairing, but he was so far beyond my abilities at the time that I felt bad about holding him back. I spent a lot of my days with him thinking “Well, I have no idea what he’s doing right know but I just made him stop 10 minutes ago and explain everything so I can’t really do that again… Can I?” It’s tough when wildly dissimilar levels pair – in an ideal environment you should try to avoid complete mis-matches but that’s easier said than done. One of the nice things about Obtiva is that we often have more than one apprentice at a time and there’s always someone who just graduated from an apprenticeship so our apprentices get some exposure to pairing with someone close to their level.

I still have the foldable train-friendly bike, but I probably haven’t ridden it since that summer. Wanna buy it?

My Apprenticeship - Friday, June 11, 2004

Sat, 12 Jun 2004 00:00:00 +0000

I just wanted to point out that I’m not editing my original posts except where explicitly noted so all of this is really what I typed up after a day of coding. In this installment, I talk about my first day at Object Mentor as an apprentice. It’s interesting to note that the “David” I refer to is actually David Chelimsky – lead developer of RSpec. Of course Micah Martin is Bob Martin’s (Uncle Bob) son and the guy who took a chance on me for the summer.

Today was my first day at Object Mentor. It took about 90 minutes to fight through traffic in the morning, so I think I’ll be looking into some sort of mass transit.

Micah is on vacation in Branson, MO (he’s been really nice to me, so I probably shouldn’t make fun of Branson… much) so I walked in not really knowing what I would be doing. I was introduced to Susan and Telisha (secretaries), Lowell (seems like someone who’s been with the company for awhile) and David. David will be working with me until Micah gets back, but David was only hired a week ago and he has been attending one of OM’s classes for that week. So although he’s an experienced software developer, he doesn’t know a lot about the internal working of the company. I guess we’ll be learning together.

We got me logged into the network and David started to explain what my project was going to be. Apparently there is this testing program called Fitness which has a wiki (a community run, although moderated, website which can quickly be updated) which is written in Java (a programming language). But there is also an OM wiki which is written in Python (another programming language). So I’m to translate the Pyhton stuff into Java stuff. I think. Since this isn’t David’s project he wasn’t super clear on the details.

But I have only heard of Test Driven Design (which is what the Fitness program is all about) so we had to back up at this point and give me a little intro. The Bowling score intro. So if your goal is to write a program to score a bowling game based on the pins knocked down during all the rolls, you could just start coding. But that would be silly. According to TDD, you should first write a simple test of the program, which will of course fail because there no code yet. Then you should write the simplest possible code that will enable your program to pass the test. So we started with a test for all gutter balls: The score should be zero. So I wrote some code that would enable the scoring program to pass that test. Then we moved on to a test for always knocking down 1 pin, the score should be 20. Fine, I got the code to pass that test. Then I ran into some trouble.

Before I go into that, I should explain why TDD is good. Or at least the short version ‘cause there’s whole books on this subject. Simply put: TDD forces you to write simple working code. Simple is good because you want your code to be readable after you walk away from it (the maintainability thing). Working is good for obvious reasons. As your code gets more complex, you’ll need to go back and change things which will inevitably cause something unexpected to break. If, after every change, you run the tests you will find out about problems sooner (before they get buried in all sorts of other changes all of which could potentially be the problem) and the test will tell you exactly what’s not working which is a tremendous help in debugging. Cool huh?

But I have no idea how bowling is scored. Which is a bit of a problem. Strikes and spares are weird. And that last frame… So I ran into difficulty. David went back to his class while I struggled with the strike and spare thing. And it was hard not to start worrying that I was in over my head. I found myself thinking “Look, this isn’t that hard so why can’t you do it? Hmm? Maybe you don’t have a future as a programmer. Maybe they’ll realize their mistake in taking me on and get rid of me. Or worse, they’ll be polite and assign me inordinately simple work so as not to tax my feeble brain.” Yes, I know this all seems silly. But it was my first day and, frankly, I have a lot riding on being able to become a developer so it was hard not to feel a little panic. After a while David came back, saw I was struggling and gave me a hint. After that I got it.

Somewhere in there Lance (salesman) came in with his niece. In between making plans for the Agile/XP universe conference he mocked his niece for not being able to manage a turnstile on the ‘L’. She ignored him. I ignored both of them as I was wound up in my impending failure as a developer.

But I worked through that, as previously discussed, and we moved on to the “so you don’t know Python” part of my day. David showed me how to download Python and set me to work figuring out a Python tutorial. This took up the rest of the day. Python is weird. Yes I know everybody loves it, but those “for” statements and their use of “lists” are intelligible but strange. I think I might try re-doing my TicTacToe program in Python as a learning exercise. But that depends on how ambitions I am this weekend. My wife’s play is opening Friday (more on that elsewhere in this site) and then there’s all those videogames I just bought (T.H.U.G., Pandora Tomorrow, and True Crime: the Streets of L.A.). Tough choices.

Wow. I forgot that this was the first time I was seriously introduced to TDD – and from David Chelimsky no less. What’s funny is that TDD was still pretty new to him. Also, I started on a Friday because finals had wrapped up on Thursday (and I had graded them in a rush). I didn’t even take a day of summer vacation – that’s kinda crazy now that I think back on it.

Also, also, I really did freak out of my mind when David gave me that bowling problem (which, btw I now know is a classic example of TDD) because I totally had no idea how to score a bowling game. And I was way too scared to ask. And later I was ashamed that I hadn’t asked earlier. Eventually I had to just suck it up and confess that I had no idea. David was really nice about it.

My Apprenticeship - May 2004

Sat, 15 May 2004 00:00:00 +0000

Prologue for the prologue: In 2009 I revisited my original Object Mentor Apprenticeship posts from 2004. Since then I’ve merged the commentary and the originals. All of the posts about “My Apprenticeship” contain my original 2004 content and have indented 2009 commentary. Additionally, I apologize for a dumb “Larry the Cable Guy” joke arriving from the past shortly.

It’s the 5 year anniversary of my 3 month apprenticeship at Object Mentor so I thought I’d revisit my original blog posts about my time there. Why? Well mostly because I used to collect LaserDiscs and really enjoyed listening to a director’s commentary on something they had done many years ago. These days commentary tends to be done either right after the movie is made or because there’s a financial obligation and it kinda cheapens the whole thing – I’m hoping I can at least do better than “Larry the Cable Guy.”

Anyway, this first post was written in May of 2004 before I actually started working at O.M. And since it was the first post I think I did a pretty good job setting up my background. I should point out that I was a high school physics teacher at the time and so I was giving up my summer in a desperate gamble to get a new job. Most people, when I told them I intended to become a programmer without going to college, thought I was nuts but were nice enough to nod and smile. I was pretty sure it was a long-shot myself but I had just started teaching “remedial physics” which is physics for kids who don’t know algebra and have problems with calculators so I was ready to try just about anything.

And I did. Anyway, without further ado, here is my first blog post ever:

I’m going to be spending the summer at Object Mentor learning various programming languages, agile software development, and all sorts of other stuff. In return for dealing with me, they get free labor. All the monkey-work that would be a waste of a good programmer’s time will be handled by me. No, I’m not exactly sure what that will be. Should be a pretty exciting summer.

First I should talk about my background and where I am now as a programmer. Probably the first computer I ever used was a TRS 80 in grade school. I was part of some sort of gifted pull-out program and I wrote the usual things a 4th grader would (10 print “Jake is cool.” 20 goto 10). During the 80’s, when I was in junior high, I got my parents to buy me a TI 99 4/a and I learned Basic. I spent a fair amount of time writing programs. Extended Basic could do sprites, but was so slow that I could never get any decent video games to work because the computer never caught any collisions. All my lasers passed harmlessly though their targets.

Getting a compiler so I could write something faster cost just about as much money as the whole computer, so I was stuck with Basic. Pretty soon I got bored with it and moved on. In high school I got interested in physics and later, in college, I decided to become a physics teacher.

Now here it is 20 some years later and I’m back learning to code. Where’d all the line numbers go? And what’s an Object?

About November of 2003 I started working through Eckel’s “Thinking in Java.” Good, but tough for someone who doesn’t know C. In January I started taking an Intro to programming in Java class at a local community college. This was pretty helpful, but also pretty easy. It gave me good footing in what modern programming is all about.

Since I wasn’t going to start at O.M. for about a month and I was anxious to learn something, Micah (Micah Martin – the guy who will be looking after me during my apprenticeship) gave me a project to work on: Design an unbeatable Tic Tac Toe program.

My first pass at this went like this: A TTT board is basically a 3X3 matrix (array), so how about empty spaces being 0, X’s are 20, and O’s are 1? This way I can multiply all the numbers in a row (or column, or diagonal) to see if there’s a win (any zero’s will make the product zero, and 3 X’s or 3 O’s will produce a distinctive number). So checking for a win is easy. So is checking for offense or defense. Add up the row (or column, or diagonal) and a potential win will be either 1+1+0 = 2 or 20+20+0=40. So the computer can block wins, or win itself if it sees the chance. But, there are some ways to set up two winning moves at once, so I needed a series of conditional statements to handle the first few moves (kind of like in a chess program when the computer is “on book” – playing out well researched moves). If there were no preset moves, offense, or defense, then the program made a random legal move. Well, it worked, but the code wasn’t very elegant. Here it is: (Note: Future Jake moved the code to GitHub. The “first_pass” dir contains the code I am referencing here.) Micah had me clean up the code lots and add the ability for the computer to play itself. He also wanted the human to be able to chose between X or O. Since I wrote the program with the Human always being O, this was a pretty big re-write.

After we got that cleaned up, he had me look up the minimax algorithm and use it to make the computer move. Basically minimax has the computer create a game “tree” with each possible move being a branch. The leaves are the end points: win, lose, or draw. Winning is scored +1, losing is scored -1, and a tie is (kissing your sister) 0. The computer picks the maximal score if it is the computer’s turn, but it picks the minimal score if it is the opposing player’s turn (based on the assumption that the opponent will play perfectly, and that he/she/it wants to win). Now if the board is empty, the computer has to look at around 40,000 moves. So the first move takes awhile. But the cool thing is that now the computer not only doesn’t lose, but it can set traps for the other player. Neat.

This algorithm uses recursion to construct the game tree, and that was pretty new to me (I had used some trivial recursion before, but nothing this involved) so this re-write took a fair bit of time. Here’s the code: (Note: Future Jake moved the code to GitHub. The “second_pass” dir contains the code I am referencing here.)

Doing all that filled up the month before I started nicely (keep in mind I was still teaching full time, so this was free time work).

Isn’t the part where I talk about multiplying or adding up the TicTacToe rows weird? Where the hell did I come up with that? I seriously have no idea.