Archives

Users.each do |user|

  puts user.name

end

Ever wondered about how expensive a simple “.each” on an ActiveRecord object collection is. This essentially means for every iteration of the loop, there is going to be a database fetch. Imagine looping over a collection that has 10,000 records (forget millions..). Yes, I have made use of this very same thing a bunch of times without realising the kind of performance implications this could have. Not until last week I discovered that there was a very less-expensive way to do this.

I was perusing the Rails guides casually looking for something else when I hit upon this. “.find_each”. What does this do? It makes looping over a collection of Active record objects much much cheaper by performing a batch retreival of the database records and caching them. The batch size can be altered by passing in a batch_size param like this “.find_each(:batch_size => 500″).

Users.find_each(:batch_size => 200) do |user|

  puts user.name

end

I have almost stopped using “.each” these days and let “.find_each” do the job.

Between 0.9.26, RC and final 1.0 release, bundler went thru some heavy changes with respect to command line options it supports. Luckily, twitter, forum and blogs kept everybody in the loop.

The blogs, however, became obsolete quickly and even blog posts few months old don’t work with latest version of bundler. We came across this issue that kept is in a loop for a while.

Snow Leopard changed a few things with ruby and mysql gem when it came out. Most mysql installation issues are hammered out by now and are well documented. In summary, installing mysql gem requires 2 things:

• Specifying architecture flags: ARCHFLAGS=”-arch x86_64″
• Specify location of mysql_config to compile agains: –with-mysql-config=/usr/local/mysql/bin/mysql_config

photo credit: Felix Francier

I’ve been using Git for close to a year now, and I cannot remember what I did before it. I literally forget the process involved in doing certain things with subversion, cvs, and (shudder) vss. At first it was a bit of a struggle sure. The gui tools are somewhat lacking, but what is there is decent enough. I like Gitx a lot for what it does, but mainly I use it for partial commits, and visualization of a source tree. Being from windows land, and having used tortoise svn (a very very nice subversion client), it took me a bit to master the intricacies of the git command line interface. I still get into a little trouble from time to time. But it is all so very much worth it!

git branch my_new_toy

Seriously? That’s it? And it took less than a second? Why would I not branch ALL THE TIME. Wanna try something out? Branch it! Unsure that you’ll finish a feature in time for iteration end? Branch it! My pet project has 5 branches going right, all for little experiments and features that I’ve been slowly working on.

Feature branches have changed my workflow entirely. Every new feature I build, I create a branch for it. If it takes a while I merge from master to make sure I’m up to date.

git merge master

If we balk at the feature, or I want to go a different route with it, I can chuck it. If all goes which (which of course it always does), I merge it to master or whatever branch represents the latest greatest code.

git checkout master
git merge my_feature_branch

Once you get the hang of it you realize how simple and powerful it is. Every day I use it I’m continually amazed and thankful of how well it works and how drastically it improves my teams workflow.

photo credit: the_moog

We develop digital products. Lots of them. So when I talk about things that are hard or easy to do, it’s based on hard won experience.

Two products that we’re about to help clients launch provide a perfect contrast on the two kinds of “hard” you find in software product development.

First we have a data visualization app that cranks through 72 million point datasets and cranks out 72 vector graphics surfaces, each with some 6000 curves comprised of a quarter million points. This is then displayed in real time to an expert audience of specialists via a highly responsive interface. We’ll call this one “dataviz.”

Second, we have a healthcare application that allows healthcare professionals to conduct a guided interview with a patient, both capturing information and educating them via feedback. We’ll call this one “interview.”

After working in ruby on rails for a number of years, I have found a number of performance related bottlenecks due to a lack of indexes on the database tables. I’ve noticed that most rails developers don’t write much SQL anymore, which is great, but also sometimes a problem. Back in the day, you really got your hands dirty in SQL, which made you really think of tuning the database. As relational databases are still around, I figured I’d give an overview on index basics.

I am going to summarize my findings here, and let those of you interested keep on reading through my empirical findings….

New Developer: When should I use an index?
Me: Any time the column appears in the WHERE, GROUP BY or ORDER BY clause.

Index Rules:
1. if you add a column for a belongs_to (like t.integer :user_id) for belongs_to :user – make sure there’s an index
2. if your column will appear in a WHERE, GROUP BY or ORDER BY clause
3. try to use composite indexes to reduce the amount of indexes needed.
4. other than that – wait until your app is up and running, and the use some performance tools to identify a need for other indexes.
5. Generally – use a full text search engine like solr or sphynx if you are getting into full text searching. Even if your database supports it, the SQL needed doesn’t work the same way on all database vendors.

photo credit: Wootang01

The NYT had a very interesting article back on September 6th, entitled In a New Role, Teachers Move to Run Schools. I didn’t have time to blog about it, but wanted to come back to it  since it rang a lot of agile bells. The idea is that teachers, rather than principals, make decisions about how to run a school.

Hmmm, self organizing teams, continuous improvement, motivated teachers? Sound a little like agile.

From the article you’re seeing some things in this Newark school that you’d consider unlikely to happen in a more traditionally run school: longer school days, teacher crafted curriculum, teacher budget governance. Self organization leads to greater motivation and job satisfaction. As one teacher put it, “This is a fantasy. It’s six passionate people who came together and said, ‘Enough is enough.’ We’re just tired of seeing failure.”

When starting a new project, we employ a technique we call Inception.  At a high level, the goal of Inception is to develop a shared vision for the project, come up with an initial scope (user story list), provide an updated estimate, and identify areas of risk.

A pitfall that is challenging to avoid is getting stuck in analysis paralysis.  Our customers love to dig into the details of how each story could work.  Our designers are chomping at the bit to get creative on the whole user experience.  The developers are, quite naturally, afraid to estimate with only a story phrase and discussion to go on.

So how do we balance the tension between the desire for ever increasing levels of detail and getting thru with initiating the project and onto the value-added activity of building the product?

First, we get the whole team on board with what the goals of inception are (as stated above)…and equally important, what they’re not.  The goal of inception is NOT to gather all the requirements nor is it to design the whole UI.

Next, we use the many of the same organizing practices and rituals we use during development: time-boxing, daily stand-ups, retrospectives, a backlog, etc.

I think time-boxing is probably the key to avoiding the analysis paralysis trap.  For example, let say we’re going to determine the scope of the admin portion of an application.  We might schedule one or two 1.5 hours workshops with the goal of mocking up a UI and culling a reasonably thorough story list.  With a strong, experienced facilitator, we complete the analysis in the time allotted.

It can be a bit scary and uncomfortable.  We will have missed requirements and when it comes to estimating, some of our assumptions will be correct, while other will prove to be wrong.  But we will get started building software and the customer will start to see their dreams coming to fruition…..and we all learn and adapt along the way.

Cross-platform, the promise of internet standards. Well, sure with enough [if IE] statements!  With so many browsers and OSs around, everything needs to be tested hands on – there are no guaranties in this fast changing environment.

Like most developers, I work on a single machine and I can be at the office, at home and where ever there’s an internet connection. I find it a great hurdle to lug around additional OS images and to have to configure a new one every couple of months. I also find it a significant expense for a company to maintain all the posible VM environments just for testing.

Some good people have taken up this problem and since I found out about them I haven’t looked back.

Brianna Sylver, Founder and President of innovation research and strategy firm Sylver Consulting, engages a lego sprint with her team. Photo by Nate Burgos

The lego blocks returned with gusto, in greater numbers and colors. This time, the participants were members of ChiPMA, the Chicago Product Managers Association. The group of thirty-three were divided into five groups. Each acted as a dedicated team responsible for developing and delivering the product, in this case, an animal with enclosure. Before going through three quick rounds of energetic lego shuffling and assembling, the group estimated and prioritized user stories and consulted with their product owners, played by members of the Pathfinder Development team.

Pathfinder’s Sr. Agile Project Manager/Coach Glenn Bernsohn led the practical session demonstrating the rhythm of an agile project. ChiPMA participants learned firsthand a number of its key concepts like Iterations, Testing, Velocity and Retrospectives.

As ChiPMA member Tim Perry put it, “Was a really fun game. Great way to meet new people and learn more about agile in a ‘non classroom’ setting.”

See the rest of the lego fun-charged pictures (slide show).

For more lego exploits, see our previous post about the game played by members of the Chicago Agile Project Management Meetup Group.

photo credit: ˙Cаvin 〄

Daniel Markham is seriously pissed off about Agile. In his most recent blog post, Agile Ruined My Life, he takes off his belt and beats the crap out of all those agile loving hippies.

But seriously, there does seem to be a backlash against agile development methodologies in some quarters. If you read Daniel’s post carefully, though, you’ll see that despite the sensational title, it isn’t really a rant against agile development, but rather an indictment against imposing agile practices in a corporate, top down way.

I’ve had the pleasure of working in a company that I started more than 11 years ago, and from the beginning we tried to practice agile development. But not everyone was on board in the beginning, and, aside from myself, there was little culture or buy in. It took us quite some time to bring in motivated “A” players with agile experience. (Thanks, competitors, for trying and abandoning agile, leaving us with a pool of qualified developers, product designers and PM’s.) It took us even longer to hone and productionize our process to where it is flexible, reproducible and highly efficient.