Tod means Fox

Business Intelligence, Data Warehousing, SQL, Visual FoxPro.

Archive for February, 2007


Published February 20th, 2007

I am King

Today at work they held their annual Mardi Gras party. Of course there was cake and other sweets so I was there. Anyway, the tradition at these parties is to crown a king. The process is simple: find the gold coin (well, nowadays it’s a tiny plastic baby) in your slice of the cake and you’re it. Guess who found the baby?

Yours truly.

So now that I am king, I have a few rules. They are as follows:

1.) I will now be referred to as King WCH (or King Hero, if you prefer)
2.) My wife is Queen Noor, and will be referred to as such
3.) No more stupid plastic babies in the cake. I want gold: real gold.
4.) Global warming is real and all those before me will acknowledge this truth

5.) Chocolate is good for you. The debate is over.

Being king comes with a catch: I need to host the party next year. I had better start planning!

Published February 7th, 2007

Ragged Hierarchy Alert

So this week I was faced with writing yet another hierarchy script. I say this with affection, of course. I find that developing efficient hierarchies can sometimes be quite a challenge depending on the data set. When there are only a few records (a hundred or so) almost any method will work (SQL, procedures, or a mixture). As the data sets grow, the method becomes more and more important. Couple this with the hierarchy requirements (can a child have more than one parent? Is there a maximum depth? Will the result be used to populate a tree control? Do I need to process each branch now or can I just get the top level nodes? Etc.) and the tidy and efficient code you imagined is now becoming bloated and difficult to debug.

There are two ways to store hierarchies in a database: Using a recursive pointer or a bridge table. The recursive pointer is possibly the most widely used. Each record has a primary key and a foreign key that points to the primary key of another record in the same table (more on this in a bit). This approach is simple to maintain and intuitive. The bridge table provides much more flexibility (for example, a child can have more than one parent), but is more difficult to maintain (you do have another table to update, and there could be issues with time stamps and sequences if a child can have multiple parents). The discussion on bridge table hierarchies ends here. I may address it in a future blog, however.

The key to ragged hierarchies is that any record in the table can participate to any depth or degree. Organizational hierarchies (which I will demonstrate in a moment) are very common and take on a ragged frame; they are also perfect for the recursive pointer. The following figure will give you an idea of how a ragged hierarchy might look:

Like I said earlier in this post, there are lots of ways to snake through this diagram: from using the OVER clause in SQL Server 2005 to writing a procedural program in FoxPro, the possibilities are as varied as the structure. In my VFP applications, I always find that a mixture of SQL, some procedural code, and a recursive function provides the most flexibility and greatest performance. To demonstrate, consider the following:

CREATE CURSOR Employee (emp_id i, emp_name c(25), sup_id i)
INSERT INTO Employee VALUES (1,"Martha Jones",0)
INSERT INTO Employee VALUES (2,"Dan Brown",1)
INSERT INTO Employee VALUES (3,"Ed Smith",2)
INSERT INTO Employee VALUES (4,"Sarah Parker",2)
INSERT INTO Employee VALUES (5,"Henry Johnson",1)
INSERT INTO Employee VALUES (6,"Samual Smyth",5)
INSERT INTO Employee VALUES (7,"Ali Jennah",1)
INSERT INTO Employee VALUES (8,"Tori Heart",7)
INSERT INTO Employee VALUES (9,"Bob Jones",7)
INSERT INTO Employee VALUES (10,"Kelly Robinson",7)
INSERT INTO Employee VALUES (11,"Manny Diaz",7)
INSERT INTO Employee VALUES (12,"Jack White",11)
INSERT INTO Employee VALUES (13,"Melinda Jo",11)
 
CREATE CURSOR crTree (;
      top_level l,;
       this_id i,;
      description c(25),;
      parent_id i,;
      deep n(2))
 
*-- run the recursive function
SELECT Employee
LOCATE snake(0,0)    && start at the top 
 
FUNCTION snake (tnParenID, tnDeep)
 
  LOCAL lcAlias
  lcAlias = "crTemp_" + TRANSFORM(tnParenID)
  SELECT * FROM Employee WHERE sup_id = tnParenID INTO CURSOR (lcAlias)
 
  tnDeep = tnDeep + 1
 
  SELECT (lcAlias)
  SCAN
    put_record(&lcAlias..emp_id, &lcAlias..emp_name, &lcAlias..sup_id, tnDeep)
    snake(&lcAlias..emp_id, tnDeep)
  ENDSCAN
  USE IN (lcAlias)
 
ENDFUNC 
 
FUNCTION put_record (tnThis_id, tcDescription , tnParent_id, tnDeep)
  INSERT INTO crTree (;
      top_level  ,;
      this_id    ,;
      description,;
      parent_id  ,;
      deep   ) ;
  VALUES (;
      tnParent_id=0,;
      tnThis_id    ,;
      tcDescription,;
      tnParent_id  ,;
      tnDeep )
ENDFUNC

I start off by creating a dummy Employee table, followed by a tree cursor (crTree)
to hold the results of my work. Next I call the recursive function ’snake’, whose job it is to find each child of the passed in parent, tracking its depth as it goes. The assumption in this program is that there is at least one zero sup_id key for the top-most level. This code works well to populate treeview controls (you could easily add a unique character sequence ID to each record). You can also output a text representation:

FUNCTION show_tree()
  SELECT crTree
  LOCATE
  SCAN
    ? REPL("-",crTree.deep) + " " + ALLT(crTree.description)
  ENDSCAN
ENDFUNC

There you have it! For 1000 employees, the snake function takes about .8 seconds to run. Anything more than that and you should consider populating the tree only to a certain level (perhaps all parents and one child deep) and filling in the rest as needed. Another approach would be to turn this cursor into a table, relate it to your Employee table and update it as part of a save routine.

Published February 4th, 2007

Spam and Rabbit Stew Lounge

This post has nothing to do with FoxPro, but I just wanted to take a moment for a personal matter. Due to the tremendous amount of spam seen at my site Rabbit Stew Lounge (I am the publisher and designer), I have implemented an invite-only system. Considering it is very unlikely that spammers will do their homework and sniff around for invite codes, I thought I’d post a special one here for anyone interested in the Fox community. The code is VFOXPRO1.

Well, here is some foxpro stuff after all:

Rabbit Stew Lounge is a non tech site. Most conversations are about current events, politics, music, and food. There is a blog feature, a forum, and wiki. I use VFP to connect to mySQL to administer the database. I had dreams of building a complete control panel in VFP that can access each feature of the site, administer users, read the RSS feeds, etc. I’ve done a lot, but it certainly isn’t in ‘control panel’ form. Perhaps as time permits I’ll get more involved again.

Published February 1st, 2007

An Introduction to the Coverage Profiler Application

Visual FoxPro (starting in version 7) comes with a built-in tool called the Coverage Profiler Application. This tool is designed to both cover (log information about each line of code run during program execution) and profile (produce useful information about the lines of code that are run) source code. The primary use for this tool is to detect code bottlenecks (which lines of code are taking too long to execute) and to locate lines of code not executed. These two functions of the Coverage Profiler are worth their weight in gold.

Sometimes I get the feeling that software performance is an afterthought during the development cycle. The code must work first. If time permits, then it is customary to go back and fine tune the source for performance. But reality (which many of you are living in) soon sets in, and time just doesn’t permit for this type of engineering. Of course, if performance is absolutely horrible then that must be addressed, but a half a second here or a quarter second there isn’t going to stir any milkshakes.

But coverage logs can really help. Consider a process (like saving a record) that zips along on a development machine, but crawls on an older test machine built some time before iPods came out. Having some knowledge of the coverage profiler can not only identify the problem, but can also help you solve it. You can identify the trouble spots in the call stack right down to the line of code causing the problem. Each problem you encounter will be different from the last, but identifying the problem will be the same.

The other major benefit of this tool is seeing which lines of code did not run. For testing purposes, this ability can really go a long way in flexing a new function or in testing a ‘fixed’ bug. It can show you where test cases are falling short, and allow you to write updated scripts that touch each line of code. It will also expose unnecessary blocks of code that an eager developer might have thought necessary (Developers: when was the last time you added an OTHERWISE statement to a DO CASE that you knew would never run? Of course, this practice is a good idea for clarity and readability, but perhaps a comment or ASSERT would be sufficient?).

What’s more is that the source code for these tools (coverage and profiler applications) is included with VFP. The Fox Development team has always encouraged its users to modify and enhance these tools to suit individual needs. Unfortunately I have not seen too many enhancements online over the years from other developers. Maybe the core functionality is good enough? I think so. But Perhaps I’ll think of something to build and put it here on this blog…