Archive for category Software Development

Live from FoxForward: Day 2 (Maps, Services, and Tiers…oh my)

Day two was a great day. I find that the size of the conference is perfect for the technologies being discussed. A lot of attention is given to the decline of conference sizes over the past 5 or 6 years. This is true for VFP and other conferences as well. But clearly, the number of “heads” do not diminish the worth of the presentations or the work that the speakers have put into them. I like the intimate atmosphere and will be looking forward to attending more conferences like it.

Fox Trails

Brian’s Fox Trails presentation was great. I didn’t know what to expect (even though I had heard of Fox Trails, and knew what it was supposed to do). I’m not a Ruby guy, and I’ve honestly never evaluated the technology. But his Ruby-borrowed techniques for accessing data and functions in FoxPro was eyeopening and insightful.

One of Ruby’s tenets is: “Don’t repeat yourself”. Brian was quickly to repeat himself… and he almost did. A hearty laughter ensued… Another Ruby tenet is “Convention over Configuration”, making applications more “self aware”. Using directory naming conventions, for example, allows you to write code that can assume certain things about its environment. I’ve always been a big advocate of “Reconfigure over Recode”. I should change mine to “Convention over Reconfigure over Recode”!

He introduced us to “MVC Architecture”, which stands for Model (data and access), View (presentation layer), Controller (business logic). This architecture parallels 3-Tier architecture which many VFP developers are used to. Those of you who have evaluated or are using Ruby will get Fox Trails right away. Those of you who do not, should take a moment to have a look.

Some additional information here and here. Check it out!

VFP Service Architecture

I’m really bummed I didn’t think of this before. I’m really bummed that I didn’t hear Stewart’s session 10 years ago. Come to think of it, ‘bummed’ is not really the word.

Stewart showed us how we can use service architecture, stored on the server as a process, to pass various tasks to the server for processing. As he mentioned, there are literally hundreds of services that you can create on the server. He demonstrated the Mailslot Messaging and PDF report generation. This means that you don’t have to rely on user’s workstation settings to send mail. It also means that they don’t need a PDF writer installed to print to PDF! You configure the service on the server, hit the service from VFP, and send mail; or print a PDF using GhostScript; or zip a file; or run some encryption; or send data via FTP; you get the idea. This architecture can dramatically reduce the amount of workstation configuration needed.

Later in the day, Stewart and I sat through Stephen Bodnar’s MapPoint presentation. I think his bulb went off when mine did. We asked Stephen, “could MapPoint be used as a service”? Because of cost and installation issues, it seemed like a perfect fit for service architecture. We’ll need to read the EULA agreement though. Stephen was skeptical. (More on MapPoint in a few sections.)

I also saw some old Fox commands that I don’t think I’ve ever had an occasion to use: “Save To” and “Restore From”. “Save to” allows us to store current variables and arrays to a variable file or memo field and “Restore from” allows us to retrieve variables and variable arrays saved in a variable file or a memo field, placing them into memory (according to the VFP9 documentation). Come to think of it, I’ve probably used them before, but just can’t remember. They seem too useful!

Developing in n-tier fashion

Michael Babcock showed us how to write non-monolithic applications in multiple tiers. He went through the basics of tier development ,from 1-tier up to n-tier and talked about Logical (same process) verses physical n-tier design (when you can’t see through the tiers, like when using a com server or the Internet).

Mike used the phrase “pass the buck” often to remind us that each tier has a job, and that all tasks (business calls, UI, data access, etc) should be handled in the appropriate tier. In other words, object responsibility. This discipline is forward-thinking and allows you to scale and upgrade applications easily. If you have ever wondered what it would take to build a web UI using your business and data access in VFP, n-tier (and perhaps with some Fox Trails) is the way to go. The same holds true for .NET front ends, mobile applications, and the like.

His business object demonstrations were great, showing the group how to correctly use the business object tier. He also went through the proper methods for using the user interface and data tiers of the design. He walked through real-world scenarios of how you might use an Oracle back end verses a CSV file using the same UI and Business Object. Mike was engaging, well prepared, and his presentation was spot on.

Map Your Data

Using MapPoint, automating it, and utilizing it through ActiveX on a VFP form, Stephen Bodnar brought us through the basics of using the tool. MapPoint with VFP is very cool. He talked about his trip back from an Advisor conference in San Diego back to Detroit (they were grounded on 9/11); then showed us the map of their travels (we hear that Whil Hentzen is a speed demon, averaging something like 80mph on the trip back).

The idea with MapPoint is to use maps to show geographic information in a better context. Just like Google and Yahoo.

I’ve been waiting to see this presentation all day because I have about a million and one needs for a solution like this. Whether showing a map of property sales, or inspection routes, this tool can certainly be used in many different contexts. I’ve been preparing myself for diving into GIS applications (probably ESRI‘s tool) for this very purpose. Now I can evaluate MapPoint as an alternative, feeling confident that it will work with VFP and that it will give me the routes and maps I need for my apps.

MapPoint has its problems, though. It is not updated frequently, and all address data needs to be exact for the tool to work correctly. You need to spend about $200 per license on every machine that uses MapPoint.

I also learned a new trick. In VFP (starting in VFP7) the Object Browser can be used to create a set of #DEFINE statements which lists all the constants in a type library by using drag and drop from editor window. The only problem with this approach is that VFP truncates decimals if contained in the constant. Rick Strahl’s tool, “Get Constants” circumnavigates this known VFP bug from the object browser. His tool is here.

On to day 3, the final day of FoxForward 2007!

Tags: ,

4 Comments

The 80/20 Principle and Software Development

In the book “The 80/20 Principle: The Secret to Success by Achieving More with Less The 80/20 Principle and Software Development“, author Richard Koch describes how a minority of “causes, inputs, or efforts” will lead to the majority of “results, outputs, or rewards”.

A great demonstration of how the 80/20 Principle (aka Pareto’s Law) works can be seen through examining source code. Clearly, a small minority of code in an application produces the vast majority of business benefit to the user. Usually, this code is well defined, tested, and performs great — after all, it is the core of what the application does. Once in production, this code is low maintenance. Any changes or enhancements are usually well scrutinized and will be tested thoroughly. Your applications are bought based on this core minority of code.

Then there’s the other code.

The other code plays more of a supporting or supplemental role: It’s the code that handles your ultra-cool menu system; It’s that extra group of reports that seldom get run; It’s that cool Calendar control that seemed like a good idea at the time; It’s all the extra features that you’ve tacked on over the years. This vast majority of code contributes least to the business needs of the application, costs you the most money to maintain, and is likely to contain the most bugs.

So what’s a developer to do?

Surely bells and whistles, gold plating, and other extras help sell the product. So I am not an advocate of stripping software down so that it is only functional. Software users expect to have a somewhat enjoyable experience behind the keyboard. Boring, functional applications would look a lot like a dos window and be the subject of many scornful conversations at the water cooler.

Instead, get the most bang for your buck. Identify the 20% and expand it. Identify the 80% and depreciate it. Revisit and repeat at each release cycle. Easier said than done? I’ll give you an example:

80/20 In Action

The volume of user reports in a mature application can get out of hand. I’ve worked on Applications in the past that have had more than 100 user reports. One day I wondered (out loud to my coworkers) which reports were actually being used. No one truly knew the answer, but we all had a hunch that the answer was “not many”. So, I added a secret logging script in my report class that logged report usage to a flat text file stored in the root directory of the application. I stored the report name, the user, and the timestamp. We retrieved the file when dialed in for support services. I found that out of 130 custom reports, only 6 were being used on a regular basis at about 100 installed locations, and a whole 50% of the reports were never run at all over the test period (11 months).

Whoa. Consider now that we spent time regression testing these reports during our latest release, that we regularly trained our users on how to run them, and that they are all included and dutifully updated in our documentation. Bad management? Poor software design? Scope bloat? Or just a perfect demonstration of how the 80/20 Principle works in software development: you decide.

The solution in this example was simple: We stopped supporting the unused reports (we continued to monitor the usage logs) and would not add any new reports without careful consideration. It felt good to trim the fat and in the end, we saved ourselves a considerable amount of work.

80/20 Analysis

During the different release periods of the software life cycle, an 80/20 Analysis should be done to both identify the core minority and the excess majority. Start with actual features (User Reports, Ad hoc Reports, Custom Query Engine, the Internet Backup Utility, Etc.) and then drill down into each feature and look for low hanging fruit. Once identified, monetize its impact on your engineering, quality, and client services teams. Also, place a value (using real sales dollars if available) on the feature itself.

This kind of analysis should reveal some interesting things about the software. If there are gray areas, for example if you are unsure of what is being utilized, consider doing something like I did to track report usage. Coverage profiling engines will allow you to identify seldom run or never run lines of code. These areas might give you clues as to what code will buy your house or pull you under. You can create a simple but professional survey on your website and ask your users what features they find most helpful and which features they don’t use. Not only would this survey tell you what is not being used, but it could also help you identify areas where you can increase the 20%.

For more information on the 80/20 Principle, I invite you to pick up Richard Koch’s book at Amazon. Have a read, and share with me your thoughts!

Buy The 80/20 Principle: The Secret to Success by Achieving More with Less The 80/20 Principle and Software Development now at Amazon.

Tags: , ,

2 Comments

David Woods Learns FoxPro

David Woods talks about a recent project where he is connecting to an older legacy FoxPro system using ADO.NET. He lists out 6 “stumbling points“, which include some common errors during the connection process as well as some words of wisdom when writing queries. Actually, the best point is number 6. He writes:

6. This applies to everyone. CODE TO INTERFACES. By coding to the IConnection, ICommand, etc. interfaces my switched back and forth between odbc and oledb took seconds not days.

This is solid advice and should be followed by all OOP developers. Coding to interfaces (and not objects) saves time, simplifies designs, and increases flexibility especially among teams of developers. Thanks David for the advice!

Tags: ,

No Comments

Guineu means Fox

If you haven’t discovered yet, the Foxpert himself, Christof Wollenhaupt, is engaged in an interesting and important extension to VFP9: Guineu. It is billed as an “alternative runtime library for Microsoft Visual FoxPro® 9.0 that runs on any Microsoft .NET compatible platform”.

Hopefully I’ll be able to take a closer look by participating in some beta testing.

Oh, and yes, according to Christof:

Guineu is Catalan, a language spoken in Northern Spain, Andorra and some parts of France. La guineu (it’s female) translates to “fox” in English. When pronounced the “U” is silent and “E” and “U” are two separate vowels.

…guineu does means fox!

Tags: ,

No Comments

Newton’s Method for Approximating the Zeros of a Real-valued Function

Need to find the root of a real-valued function and simple algebra just won’t cut it? Then Newton’s method is for you (if you can remember back to your calculus days, that is).

Newton’s method is a root-finding algorithm and uses iteration to solve nonlinear equations. The process involves making a guess at the true solution and then applying a formula to get a better guess. This process repeats until the (approximate) answer is found. The methodology is part of numerical analysis, which is the “branch of mathematics dealing with methods for obtaining approximate numerical solutions of mathematical problems (dictionary.com)”. Specifically, the type of problems fit into the “continuous mathematics” category that are common in medicine, engineering, science, and business industries. (For a good discussion on continuous verses discrete mathematics, check out this blog entry by Mark C. Chu-Carroll or this page from ESC.EDU.) As you might guess, continuous mathematics is very important in business intelligence, specifically statistical analysis of data.

The general form to find the approximation is as follows:

450bac44dc3e19e718bd2aa922f3d9ba Newtons Method for Approximating the Zeros of a Real valued Function

Here’s how it works: Start with an initial guess (get out your graph paper!). Then the function draws a tangent to approximate the function based on your initial guess. Next, the function computes the x-intercept. This x-intercept will be better than the initial guess, and closer to the answer (the root). It then repeats with the x-intercept as the new guess.

The FoxPro code:

LOCAL lcFunction, lnInitialGuess, lnIterations 

*-- set the fuction, my initial guess, and number of iterations
lcFunction = "(1/4)*x^3 - 3*x^2 + (3/4)*x - 2"
lnInitialGuess = 2
lnIterations = 10 
 
*-- run newton
newton(lcFunction, lnInitialGuess, lnIterations )
 
*-- foxpro does newton:
FUNCTION newton
LPARAMETERS tcFunction, tnInitialGuess , tnIterations 
 
LOCAL lcDerivative AS String
LOCAL lnNewGuess, lnIter , x , lnY , lnSlope , lnCol5 , lnCol6
 
CREATE CURSOR crNewton (col_n n(2), col_Xn n(14,5), col_fXn n(14,5), col_f1Xn n(14,5), col_ys n(14,5) , col_ng n(14,5))
 
limh = .000001
lcFunction = ALLT(tcFunction)
lnInitialGuess = EVL(tnInitialGuess,0)
lnIterations = EVL(tnIterations,1) 
 
lnNewGuess = lnInitialGuess
 
*-- Loop for each user iterations
FOR lnIter = 1 TO lnIterations
 
  *-- Set guess
  x = lnNewGuess
 
  *-- Run Function
  lnY = &lcFunction
 
  *-- Find Derivative
  lnHoldx = x
  x = x + limh
  lnFuncpluslimh = &lcFunction
  x = lnHoldx
  lnFuncminlimh = &lcFunction
  lnSlope = (lnFuncpluslimh - lnFuncminlimh) / limh 
 
  lnys = lny/lnSlope 
 
  *-- Find New guess
  lnNewGuess = x - lnys
 
  *-- Populate the cursor
  INSERT INTO crNewton (col_n  ,col_Xn ,col_fXn ,col_f1Xn ,col_ys ,col_ng) ;
                VALUES (lnIter ,x      ,lnY     ,lnSlope  ,lnys   ,lnNewGuess)
 
  *-- Reset the guess and continue
  lnNewGuess = lnNewGuess
 
NEXT
 
SELECT crNewton
LOCATE
BROWSE NORM
*-- set the fuction, my initial guess, and number of iterations
lcFunction = "(1/4)*x^3 - 3*x^2 + (3/4)*x - 2"
lnInitialGuess = 2
lnIterations = 10 

*-- run newton
newton(lcFunction, lnInitialGuess, lnIterations )
 
*-- foxpro does newton:
FUNCTION newton
LPARAMETERS tcFunction, tnInitialGuess , tnIterations 
 
LOCAL lcDerivative AS String
LOCAL lnNewGuess, lnIter , x , lnY , lnSlope , lnCol5 , lnCol6
 
CREATE CURSOR crNewton (col_n n(2), col_Xn n(14,5), col_fXn n(14,5), col_f1Xn n(14,5), col_ys n(14,5) , col_ng n(14,5))
 
limh = .000001
lcFunction = ALLT(tcFunction)
lnInitialGuess = EVL(tnInitialGuess,0)
lnIterations = EVL(tnIterations,1) 
 
lnNewGuess = lnInitialGuess
 
*-- Loop for each user iterations
FOR lnIter = 1 TO lnIterations
 
  *-- Set guess
  x = lnNewGuess
 
  *-- Run Function
  lnY = &lcFunction
 
  *-- Find Derivative
  lnHoldx = x
  x = x + limh
  lnFuncpluslimh = &lcFunction
  x = lnHoldx
  lnFuncminlimh = &lcFunction
  lnSlope = (lnFuncpluslimh - lnFuncminlimh) / limh 
 
  lnys = lny/lnSlope 
 
  *-- Find New guess
  lnNewGuess = x - lnys
 
  *-- Populate the cursor
  INSERT INTO crNewton (col_n  ,col_Xn ,col_fXn ,col_f1Xn ,col_ys ,col_ng) ;
                VALUES (lnIter ,x      ,lnY     ,lnSlope  ,lnys   ,lnNewGuess)
 
  *-- Reset the guess and continue
  lnNewGuess = lnNewGuess
 
NEXT
 
SELECT crNewton
LOCATE
BROWSE NORM
*-- run newton
newton(lcFunction, lnInitialGuess, lnIterations )

*-- foxpro does newton:
FUNCTION newton
LPARAMETERS tcFunction, tnInitialGuess , tnIterations 
 
LOCAL lcDerivative AS String
LOCAL lnNewGuess, lnIter , x , lnY , lnSlope , lnCol5 , lnCol6
 
CREATE CURSOR crNewton (col_n n(2), col_Xn n(14,5), col_fXn n(14,5), col_f1Xn n(14,5), col_ys n(14,5) , col_ng n(14,5))
 
limh = .000001
lcFunction = ALLT(tcFunction)
lnInitialGuess = EVL(tnInitialGuess,0)
lnIterations = EVL(tnIterations,1) 
 
lnNewGuess = lnInitialGuess
 
*-- Loop for each user iterations
FOR lnIter = 1 TO lnIterations
 
  *-- Set guess
  x = lnNewGuess
 
  *-- Run Function
  lnY = &lcFunction
 
  *-- Find Derivative
  lnHoldx = x
  x = x + limh
  lnFuncpluslimh = &lcFunction
  x = lnHoldx
  lnFuncminlimh = &lcFunction
  lnSlope = (lnFuncpluslimh - lnFuncminlimh) / limh 
 
  lnys = lny/lnSlope 
 
  *-- Find New guess
  lnNewGuess = x - lnys
 
  *-- Populate the cursor
  INSERT INTO crNewton (col_n  ,col_Xn ,col_fXn ,col_f1Xn ,col_ys ,col_ng) ;
                VALUES (lnIter ,x      ,lnY     ,lnSlope  ,lnys   ,lnNewGuess)
 
  *-- Reset the guess and continue
  lnNewGuess = lnNewGuess
 
NEXT
 
SELECT crNewton
LOCATE
BROWSE NORM
*-- foxpro does newton:
FUNCTION newton
LPARAMETERS tcFunction, tnInitialGuess , tnIterations 
 
LOCAL lcDerivative AS String
LOCAL lnNewGuess, lnIter , x , lnY , lnSlope , lnCol5 , lnCol6
 
CREATE CURSOR crNewton (col_n n(2), col_Xn n(14,5), col_fXn n(14,5), col_f1Xn n(14,5), col_ys n(14,5) , col_ng n(14,5))
 
limh = .000001
lcFunction = ALLT(tcFunction)
lnInitialGuess = EVL(tnInitialGuess,0)
lnIterations = EVL(tnIterations,1) 
 
lnNewGuess = lnInitialGuess
 
*-- Loop for each user iterations
FOR lnIter = 1 TO lnIterations
 
  *-- Set guess
  x = lnNewGuess
 
  *-- Run Function
  lnY = &lcFunction
 
  *-- Find Derivative
  lnHoldx = x
  x = x + limh
  lnFuncpluslimh = &lcFunction
  x = lnHoldx
  lnFuncminlimh = &lcFunction
  lnSlope = (lnFuncpluslimh - lnFuncminlimh) / limh 
 
  lnys = lny/lnSlope 
 
  *-- Find New guess
  lnNewGuess = x - lnys
 
  *-- Populate the cursor
  INSERT INTO crNewton (col_n  ,col_Xn ,col_fXn ,col_f1Xn ,col_ys ,col_ng) ;
                VALUES (lnIter ,x      ,lnY     ,lnSlope  ,lnys   ,lnNewGuess)
 
  *-- Reset the guess and continue
  lnNewGuess = lnNewGuess
 
NEXT
 
SELECT crNewton
LOCATE
BROWSE NORM
*-- Loop for each user iterations
FOR lnIter = 1 TO lnIterations
 
  *-- Set guess
  x = lnNewGuess
 
  *-- Run Function
  lnY = &lcFunction
 
  *-- Find Derivative
  lnHoldx = x
  x = x + limh
  lnFuncpluslimh = &lcFunction
  x = lnHoldx
  lnFuncminlimh = &lcFunction
  lnSlope = (lnFuncpluslimh - lnFuncminlimh) / limh 
 
  lnys = lny/lnSlope 
 
  *-- Find New guess
  lnNewGuess = x - lnys
 
  *-- Populate the cursor
  INSERT INTO crNewton (col_n  ,col_Xn ,col_fXn ,col_f1Xn ,col_ys ,col_ng) ;
                VALUES (lnIter ,x      ,lnY     ,lnSlope  ,lnys   ,lnNewGuess)
 
  *-- Reset the guess and continue
  lnNewGuess = lnNewGuess
 
NEXT
 
SELECT crNewton
LOCATE
BROWSE NORMAL

In this run, with the equation I have defined, the approximation of x is 11.80326. If you plug this number in for x in the original equation, it evaluates to .00014. Pretty close! And that’s exactly what this algorithm is great for. Try these additional functions to get the hang of it. You can change the guess and iterations as you like as well:

  • “x + 1″
  • “x^3-12″
  • “x*(x^3)-(3/4)*x”
  • “((x^4+1)*(x^3+2))-11″

 

Tags: , ,

1 Comment

Cast(), Str(), or Transform() — Choose wisely!

It’s been said that if you can’t do something 3 different ways in FoxPro, then it can’t be done. I’m not sure where this phrase originated, but it’s stuck with me over the years. With options though, come responsibility. Yes, you can often solve a problem using several methods, but your job as a developer is to pick the most reliable and best performing of the options available. This could mean making a choice to use the REPLACE statement instead of SQL-UPDATE, MEMLINES/MLINE over ALINES, or TRANSFORM instead of STR.

Trimmed conversion of number n

For example, run this code from a program file a few times over. They are functionally equivalent, producing a trimmed string conversion of a number n:

clear
nSec = SECONDS()
FOR n = 1 TO 100000
  lc = ALLTRIM(STR(n))
NEXT
? "Run 1 with STR(): "
?? SECONDS() - nSec
 
nSec = SECONDS()
FOR n = 1 TO 100000
  lc = TRANSFORM(n)
NEXT
? "Run 2 with TRANSFORM(): "
?? SECONDS() - nSec
 
nSec = SECONDS()
FOR n = 1 TO 100000
  lc = CAST(n as Varchar(10))
NEXT
? "Run 3 with CAST: "
?? SECONDS() - nSec

You will notice right away that using CAST is by far the fastest (on average 2.5 times faster than ALLTRIM(STR())). Transform is a little faster than ALLTRIM(STR()) as well (on average 1.3 times faster using 100000 iterations as a test). If you reduce the number of iterations, this ratio reduce until finally ALLTRIM(STR()) begins to outperform TRANSFORM().

CAST is new to VFP9, so it is likely that it hasn’t made its way into a lot of your code — but perhaps some refactoring is in order! Especially in loops with many iterations or in SQL statements.

nLength conversion of number n

In the previous example, I returned a trimmed string of a number n. Consider that you want your numeric transformation to be of a particular size. What’s faster now?:

clear
nSec = SECONDS()
FOR n = 1 TO 1000000
  lc = STR(n,10,2)
NEXT
? "Run 1 with STR(): "
?? SECONDS() - nSec
 
nSec = SECONDS()
FOR n = 1 TO 1000000
  lc  = TRANSFORM(n,"9999999.99")
NEXT
? "Run 2 with TRANSFORM(): "
?? SECONDS() - nSec
 
nSec = SECONDS()
FOR n = 1 TO 1000000
  lc = CAST(n as Character(10))
NEXT
? "Run 3 with CAST: "
?? SECONDS() - nSec

Again, CAST is faster than both STR and TRANSFORM (on average 2.7 times faster). Transform (which surprised me) still outperforms STR() (at 100,000 iterations, the ratio on average was still about 1.3). But take heed! The last example here isn’t functionally equivalent: CAST will left-align the results, while STR and TRANSFORM right-align the results. STR and TRANSFORM will round decimals to the specified amount, CAST will not.

Obviously, there are many things you can do with TRANSFORM that you cannot do with either STR or CAST. In most cases, however, CAST seems to be the way to go. Using ‘Varchar’ in CAST trims your string, using ‘Character’, does not. TRANSFORM automatically produces a TRIMed string as well, unless you specify a format code.

I like the new CAST function in VFP9. I got used to using it while writing code to interact with SQL Server. But I soon found CAST sneaking into my code in various other places — often replacing ALLT(STR()) and TRANSFORM.

Tags:

3 Comments