Archives

User Testing is always good. It always yields some worthwhile and occasionally unexpected results. After the ego’s involved recover, they realize it is always better & cheaper to know sooner.

But User Testing is not perfect. If it has a fault, it is this: user testing an existing site is like watching life in a rear view mirror. It’s a reflection of a snippet of experience; a kind of shadow casting of real use.

Often, people evaluate what they have access to and make profound determinations on the basis of what they can see. This can be a very limited view of reality, and tends to reinforce that which has already been invented; to paraphrase a tautology: That which is good exists, that which exists may or may not be good, but it can be tracked within a millimeter of it’s existence.

So it has to be taken with a grain of salt. There are famous business cases of researching an existing context and making regrettable decisions. The research that Ford did as to whether or not they should have a drivers side rear door on the Windstar strongly indicated that consumers did not care. Dodge ignored thier research and added one, taking the market from Ford. What is the lesson? Once consumers saw it, and used it, they realized that it was a valuable addition. Another chapter in Ford’s seemingly endless downfall.

To suggest that the fault is the user testing is not quite the whole story. The user testing counted what was known to be countable; what it cannot do is invent, or count what it cannot quantify. However, a mix of interviews, done by objective mediators, can reveal trends that designers & developers may not have considered or may have de-emphasized in thier process.

If you’re a DBA or a Sysadmin, your flock is probably larger than you think it is.  By this, I mean, your organization, or your department, probably has more database servers than you think it does.  Probably a LOT more.

Today’s post aims to help you locate these "lost lambs" and bring them back into the fold.  I am using the example of MySQL, but many of these strategies apply to other database software.

Like any company that manages infrastructure for clients, we often run into emergency cases where a client database needs to be restored. Sometimes we’re not sure how they are being backed up, or even IF they are being backed up at all. Why? Sometimes databases magically drift from experimental to production. This points to governance issues of how and when applications and environments should be promoted to production, but it still leaves the practical question of "what do I do now with all of my lost databases?"

In many cases we end up auditing ALL of the databases at a client’s sites. What we find often surprises the client. In one case we found more than 12 undocumented instances of MySQL! Many of these had sprung up in the past few months.

MySQL is a very capable, free, open source database server. These qualities make it very easy to bring up new instances of MySQL quickly. The downside of this is that they are often not tracked, managed, secured, or backed up as carefully as they should be.

If you find yourself in a similar situation, the first thing you should do is audit your database servers. Count them, but also determine the following information about them. (It may be helpful to use a table or spreadsheet.)

hostname

type (standalone, replicated, cluster)

IP address(es)

purpose(s)

criticality of the database server (impact if this server goes down)

criticality of the data (impact if amount x of this data is lost)

applications it depends on

applications that depend on it

security policy regarding the data and the server

privacy policy regarding the data

table engines in use (InnoDB, MyISAM, etc.)

current primary MySQL backup method

any changes proposed for primary MySQL backup method

proposed supplemental MySQL backup method

filesystem backup method

bare-metal (disaster recovery) backup/restore method

date of last MySQL restore test

date of last filesystem backup test

date of last bare-metal restore test

Please notice that we list filesystem and bare-metal backups and MySQL backups separately. All are important, but in this post we will focus on MySQL backups.

Effective MySQL backup methods vary depending on the structure of the server, and the table engines in use. Here are some suggestions:

standalone, MyISAM: mysqldump & binary logging

standalone, innodb: innodb Hot Backup & innobackup Perl script (automates a "sharp" backup of MyISAM & InnoDB tables)

replicated, innodb: stop replication on one slave server, back up that slave server using innobackup and/or mysqldump, re-syncrhronize the slave server

clustered, ndbcluster: "online backup", which is a "hot" or "live" backup initiated using the MySQL cluster management server

Also, some open source and commercial filesystem backup solutions can provide agents or plug-ins that allow "hot" backups of MySQL. One solution we often use is Arkeia. From Arkeia’s home page:

"The Arkeia MySQL plug-in is easy to install and configure: databases and table backup/restore can be selected via the convenient navigator in the Arkeia GUI. Backup can be defined up to the table level. Binary logs are also backed up, enabling the restore of databases at an exact state between two backups."

"Arkeia has a distinct advantage over other  database backup solutions: a multifl ow system that enables faster  backup than other software. This allows back up of multiple tables at  the same time."

Some people advocate LVM2 snapshots as a way to backup live MySQL databases. However, this approach can be problematic. That is because the very databases that probably need backups most (large, high traffic databases), are the ones for which the snapshot or the number of locked tables may quickly grow too large to make snapshots practical.

Mike Kruckenberg has an excellent post discussing these issues.  He concludes that IF you are using the innodb engine, and IF you prefer not to purchase the innodb hot backup program from Innobase Oy (now owned by Oracle), LVM2 snapshots may be a good choice.

We hope we have given you some food for thought. Happy Shepherding!

Technorati Tags: database administration, system administration

My review on Monday of the Sprajax AJAX security scanner left me thinking about the limitations of the old school crawlers and spiders. In order to spider an AJAX application you would need to comprehensively model the behavior of a browser. That’s more than just embedding Rhino or Spidermonkey into your scraper. You’d pretty much have to embed the whole browser and use it to identify all valid events that an XHTML document will accept at a given point.

The complex application state of something like an AJAX word processor could make it difficult to know when to stop spidering, i.e. when has our URL graph cycled back on itself.

There is at least one open source tool that allows one to at least record and run tests in the Firefox browser: Selenium. This terrific post by Grig Gheorghiu off of his Agile Testing blog explains some of the subtleties of using Selenium with AJAX. The tool may be a good place to start to build AJAX capable spiders and crawlers.

My last post in this series was on the Fact Harvest pattern for using business rule engines. My intent with these patterns is not to give some deep architectural or design insight, but to help BRE novices pierce the jargon fog that has them shaking their head on how to use these beasts. This time we’ll be talking about "The Transformer."

Pattern Name: The Transformer

Synopsis
Suppose you want to make many transformations on a dataset but don’t know all of the details of what you need to do up front. You could use a Little Language or Interpreter pattern to allow you to compose various transformation operations dynamically. Making such a solution perform would involve a fair amount of work. Instead, you could use a BRE to perform the transformations efficiently without having to develop your own domain specific language.

Context
Take the case of an ETL (Extraction, Transformation and Loading) process. This is the sort of technology you might employ if you provide data analytics for a particular industry (automotive, health care, insurance, etc.) and you ingested huge amounts of data in varying formats. Typically you would use expensive tools like Informatica or Ascential — or the Open Source tool Kettle, have a look — to wrangle the data into shape and load it into a database of some sort.

These ETL tools provide various plugins that can be chained together in a data transformation process. You can even write your own plugins in C, C++, Java or other languages. You can embed a BRE as a transformation plugin in one of these ETL tools. Your "facts" would be individual rows of data that are modified by your rules. For example, to do some simpleminded address cleaning, we might use rules like this one:

IF dirtyrecord.ZIPCODE equals georef.ZIPCODE
AND dirtyrecord.CITY does not equal georef.CITY
THEN
cleanrecord.CITY = georef.CITY
END

If you are processing truly huge amounts of data and your processing requirements are not that complex, you might have to evaluate rule execution algorithms other than RETE for improved performance. Many of the commercial vendors provide these alternate execution algorithms; ILog, for example, has a sequential execution mode for large numbers of independent rules. As always, do your own evaluation rather than trust vendor claims.

Consequence
The various commercial and open source ETL tools do have canned logic for various tasks such as normalization or name and address cleaning and they also have transformation plugins that allow for scripting. Embedding a BRE allows you to deploy powerful new data transformation rules more quickly than with scripting or custom plugins written in a language like C. Using the pseudo-natural language capabilities of the BRE’s, it is possible to construct a DSL (Domain Specific Language) more more quickly and with far greater runtime performance than building it from scratch or using other, less powerful tools.

Again, I hope this example is useful to you and leaves one less person shaking their head and asking: "How the heck do I use a Business Rules Engine?"

My colleague Bob Moll over in the UXD blog writes about the Zeigarnick Effect and it’s implication for designing GUI’s. What is the Zeigarnick Effect? Bluma Zeigarnick was a Russian psychologist who in the 1920′s discovered that we remember unfinished tasks much better than completed ones. This memory comes from a psychological pressure to complete unfinished tasks.

Bob notes that in a rich GUI, the psychological pressure is to detour and finish short tasks, leaving a trail of longer, unfinished tasks. He suggests that we include a facility for a queue of unfinished tasks in applications so that users can easily go back and finish them.

In the web application world, we avoided this aspect of the Zeigarnick Effect by making it difficult if not impossible for the user to diverge from the page flow. Now, with the capability to use fancy modal dialogs that interact with the server, we can add optional subtasks into the flow. AJAX-powered command completion helps prevent the the user wandering off in another window to search for the right thing to enter. My favorite example is the WSJ’s selection-based search, which avoid interrupted reading to search for another term.

As you design your applications, give some thought to the Zeigarnick Effect and how you can use AJAX to smooth the continuation of interrupted tasks.

I came across an open source AJAX security scanner called Sprajax. From the Denim Group’s press release:

Sprajax is the first web security scanner
developed specifically to scan AJAX web applications for security
vulnerabilities. Denim Group, an IT consultancy specializing in web
application security, recognized that there were no tools available on
the market able to scan AJAX. AJAX allows web-based applications a
higher degree of user-interactivity, a feature with growing popularity
among developers.

Denim Group developed this innovative tool that
will revolutionize security assessments by providing a more thorough
diagnosis of security vulnerabilities within the AJAX code that other
web security scanners are not designed to read. The software then
produces a report of possible weaknesses for developers to remedy.

It’s Open Source, so I decided to download it and see what the hype was about. First, the application is written in C#. Second, you will need SQL Server to get it to run as the results are stored in SQL Server and are pulled into reports using stored procedures. Finally, the application doesn’t do all that much.

The application takes an application URL as a parameter and has three main functions.

• You can "footprint" the application, i.e. scan it for Javascript files and web services.
• You can "fuzz" the application, i.e. pass garbage into the web services to see if they will barf.
• You can display the results of the previous two functions.

Essentially, the footprint function spiders the application, detects the framework used, finds Javascript files included, and discovers web services used. The fuzz function uses those discovered web services and passes some garbage into them and sees if they returned error conditions. There are just a few problems with this tool, however:

• The tool only detects the Atlas framework. No Dojo, DWR, etc.
• It only detects SOAP web services used by the Atlas framework. No REST, no framework specific calls.
• It doesn’t scan the Javascript files for XHR calls, another place to find calls back to the server.

The fuzz function only tests robustness and in a fairly limited way, i.e. will the app go belly up when you pass in garbage data. Typically, security scanning tools test for known exploits of web application frameworks, a much more useful approach.

Finally, consider that many AJAX frameworks make use of dynamic XHTML and Javascript to drive content and navigation. This tool just sucks in the content and looks for URL’s. It wouldn’t find even one link in an Echo2 application, for instance.

Overall, there isn’t much here. It has the feel of something that a journeyman programmer put together over a weekend. I applaud the Denim Group for releasing an Open Source tool, but the hype in their statement is wildly inaccurate. You can likely write something in Perl in a few hours that would do all of this and more.

The Zeigarnik effect suggests people remember incomplete or interrupted things better than completed things. It’s human nature to want to complete a task or hear the end of a story, and when we don’t a psychological tension results until the item is completed.

The effect has an interesting implication for digital interfaces. When a user involved in a long task encounters an interruption or an opportunity to complete a quick task, there’s a good chance the interruption or new task will get acted on. This can leave a lineage of uncompleted tasks and non-linear navigation trails. 

One way to combat this that we have used in a variety of applications is to dynamically create a task queue. The queue stacks up incomplete tasks in one place so the user can get back to these tasks when he or she is ready. In a restricted domain, it’s usually possible to apply some intelligence as to how the queue is built so it reads fluidly and clearly.

Lots of articles like this one from eWeek broach the subject of AJAX and security but give few answers. The most insightful quote from the article is from the Dojo man:

Panelist Alex Russell, co-founder and project lead for The Dojo Toolkit, a popular AJAX framework, said, "It’s worth noting that the fundamental problems with browser security and Web application security haven’t changed in five years—most rely on a single root of trust, and AJAX doesn’t change that. Wider spread use of cross-domain content distribution," which is not new with AJAX, is part of the issue. "The short version is still, Don’t trust the client."

Right on. But beyond "nothing to see here…move along, move along," can we at least categorize security concerns and explain how they might apply to AJAX?

The security issues, as I see them, fall into two distinct categories: end-user and server. End-user issues revolve mostly around privacy. A user wants control over what information is revealed to third parties. The server issues, on the other hand, revolve around exposing services that a malicious user can exploit by subverting the client. (There are hybrid versions of these, such as third parties launching attacks on sites using cross domain content, etc.) AJAX has added a wrinkle to both of these issues.

On the client side:

• XHR has made it difficult for the user to know whether information about their behavior is being transmitted back to the server. Solution: give users a way of seeing what is happening with XHR and other "invisible" communications.
• It is easy for developers to overlook the need for SSL or some sort of encryption when confidential information is sent back to the server. Solution: Give the use better insight into how information is about to be transferred.
• Larding in new functionality into the browser raises the chance that security holes might be opened up. Solution: decouple the browser from the OS (IE) and institute a sandbox approach. Don’t hold your breath for this one.

Should we give the user more control over XHR, i.e. give them an option to disable certain features of the browser? With the increased complexity of these apps, even tech-savvy users might find it difficult to evaluate the security of the application.

On the server side:

• Ad-hoc code – We’ve learned to protect ourselves from things like injection attacks and the like on the server side through the use of webapp frameworks. Opening up lots of little services using ad-hoc AJAX tools can lead to repeating these mistakes. Solution: consider using a server-side component GUI framework like Echo2 or ZK to avoid this issue.
• Reliance on too much business logic in the client. No matter how much you obfuscate your code, it is still accessible to the client. This is the pitfall with client-side component GUI frameworks like Tibco GI and OpenLaszlo. Solution: you can try and fight against this by making the services they access as simple as possible and dependent on cryptographically secure session information. I just wouldn’t do anything mission critical with this.
• Application complexity – It used to be that if someone compromised your site, they would deface it, install a rootkit or grab some credit card numbers. Now, a cracker might instead modify your AJAX application to send malicious Javascript to the client. Since "view source" ain’t what it used to be with DHTML and eval’d Javascript, this sort of compromise might be hard to detect. Solution: regular regression testing against the app is probably the best you can do.

The Open Web Application Security Project has some good resources on webapp security, but they’re a little bit behind on AJAX. One of their suggested approaches that I can heartily recommed is using mod_security, sort of a stateful firewall for webapps.

Design patterns, Templates, Stock designs–whatever they’re called, they present an opportunity and a challenge for designers.  All interface designers will run across this same question sooner or later.  When to customize and when to reuse a preexisting design. 
The answer to this question depends on what you consider to be the primary goal of the project.  Is it to build something unique and special–something that wows people because no one has seen it before?  Or is it to build an effective and on budget solution to a business problem? 
If it’s the former, then as a designer you’re primary goal is to create a work of art.  Go ahead and take the time to come up with something mind-boggling.  There’s no limit to what can be done, and there’s no reason to feel that a healthy dose of experimentation–both on the designer and the user’s part–won’t be beneficial.
However, if it is the later, you need to be aware that you are working under certain constraints.  Therefore the more you can reuse, the better.  There’s plenty (and I mean plenty) of existing intelligence on interface design.  It’s your responsibility to use that existing knowledge.  Incorporate it into your project.  Make use of the patterns that have been proven effective.  Make use of the processes that have been shown to be efficient.  In short, don’t reinvent the wheel.  Use the collective knowledge that the web is offering you.
And you can start here:
Yahoo Design Patterns
HCI Patterns
User Interface Design Patterns

Whether you’re writing AJAX applications using enhancements to existing webapplication frameworks or adopting the new component GUI approach, you as a developer are faced with a new granularity of events. Where before you had the monster postback, now you must respond to mouse clicks, key presses, menu selections, etc.

On top of this, with a single page interface, you may have many more controls on the page. This brings on the problem of context, i.e. that not all of these controls are appropriate for a particular context such as when a particular part of a table or list box is selected or when the focus is on a particular window.

The above just scratches the surface of the sorts of challenges that you’ll discover writing RIA’s with AJAX technologies. The good news is that there are already solutions in place since these are the sorts of problems that desktop GUI developers face all of the time. For example:

• Enabling and disabling menus and other controls can be done using the State pattern. A collection of state objects would be used to enable and disable the controls as appropriate.
• Many of the component GUI frameworks implement an Observer pattern for event management. UI events come in now in little bites instead of one huge request and can be consumed by listeners that wait on a specific component.
• MVC needs an upgrade when you’re dealing with big heirarchies of components. Combining the Composite and Chain of Responsibility design patterns, you get the Heirarchical MVC (HMVC) pattern. (If you’ve worked with WebWork you may have run across the "Pull HMVC" pattern. That heirarchy refers to the model, not the whole MVC.)
• The Command pattern is often used to implement undo.

That’s just a sampling of desktop GUI solutions you can make use of. What can you do to expand your solution space for RIA’s? Read lots of code. Find as many good open-source desktop GUI’s as you can and read their code. You’ll find ways of doing things that don’t resemble the typical webapp approach. Here are a few to get you started.

Resources:

• The Spring Rich-Client Project. Based on the Spring Framework. Used for buidling desktop GUI’s. Read the code and get a sense for how to build your RIA framework. Good source for ideas

• HMVC framework and tutorial. View/Implementation independent. You can plug this into your own apps or just learn from it.

• The Scope Framework. The bad new is that this is a desktop GUI framework based on Swing that hasn’t been updated since 2002. The good news is that this is a great place to read some code and learn how to develop component GUI applications.
• The HMVC Framework from Crionics. Again, no longer maintained, but a great place to read some code.
• Tutorial (in C#) on implementing undo with the Command pattern.
• A discussion of the Presentation-Abstraction-Control (PAC) architectural pattern. This one is just as old as the MVC pattern, but since the MVC pattern was intended for simple interfaces it was also simpler to implement. Thus MVC won out even in cases where PAC is more appropriate, such as with more complicated UI’s.

Update 1: Oliver Steele has an excellent overview of MVC as it exists in the desktop, webapp and RIA worlds. He points out why non-RIA, i.e. traditional webapp programming is so tough:

That complexity includes the real world application of what are still
research topics in academia: staged programming for multiple-target
code generation (Mac IE, Windows IE, Netscape, Mozilla, Safari, Opera),
process migration, and (usually manual) CPS conversion to maintain program state across pages.  No wonder web programming is hard!

Well worth a look.