Article: Data Lookup Benchmark »
FERDY CHRISTANT - DEC 3, 2004 (06:57:34 PM)
This article describes a bechmark that measures the performance of three common ways to retrieve data in Lotus Notes:
- Using @DbLookups
- Using cached @DbLookups
- Using profile lookups
I realize this discussion is rather old, and perhaps many have gone before me with this test, yet I still find developers (including myself) not always using the most effective method to retrieve data. This articles tries to prove which method is the fastest through a benchmark.
Before doing any benchmark, it is important to describe the test method and environment, as doing the same benchmark in different conditions may give different results. Let's sum them up:
- The test is done on a big client machine, plenty of memory and speed is available
- The data to look up is on a remote machine, to maximize the difference between caching and not caching.
- The test measures the time it takes to do a certain number of lookups. Each lookup is done to a different document, yet retrieving a constant data size every time.
Let's get right down to it. Here are the results (in seconds):
- @DbLookups without cache are very slow, yet constant. You can almost calculate the result before you even do the test.
- Cached @DbLookups have a significant advantage in effectiveness over regular @DbLookups, yet the advantage quickly dissapears when we increase the number of lookups. It looks like there is a limit to the lookup cache, and it is not big.
- Profile lookups have a significant advantage in effectiveness over both previous methods. It seems there is almost no relationship between the number of lookups, data size to look up and the measured time! The only way to explain this is that the profile is loaded into memory. Memory access speed is so fast that it does not affect the benchmark results. Both other methods have to deal with disk access and/or network access speed, which explains the slow results.
This chart sums it up quite nicely. Click to see the enlarged version:
Ok, so I should always use profile lookups? Not necessarily. Generally, you could follow these guidelines:
- Use @DbLookups without cache when it is absolutely crucial that you retrieve the latest data from the source.
- Use cached @DbLookups when the data you retrieve does not have to be the latest and greatest, yet only use it for this purpose when you are doing a few lookups (<10) with relatively small data sizes. You will have the advantage of increased effectiveness, without the fuss of profile handling.
- Use profiles when you are looking up data that does not change a lot, and when there's lots of it to retrieve. You will have the optimal effectiveness in all cases. When you are only looking up a few values from a profile, you may want to consider cached @DbLookups, which are easier to manage and debug.
Tip: Configuration data typically does not change often. In complex applications, there may be quite a lot of it. In these cases, profiles are the way to go.