Verity + Categories = Slow?
This is still a bit of a mystery, but I’m hoping to get some enlightening comments (that would be cool).
Anyway, we ran into a bit of a performance problem the other day: Verity was apparently spending over 3 seconds to perform a simple search on a relatively small collection. After a lot of time spent placing CFTIMERs all over the place, we concluded that it really was Verity. The only difference with other Verity searches I had built in the past was that we were now using the new category feature that comes with CFMX 7. So we removed the category bit from the search and we went back to a comfortable 30 to 200 ms execution time on the CFSEARCH tag…
I will now repeat. With category search: 3 seconds. Without: less than half a second.
And we were only using 2 categories in the collection so I really don’t know why this happened. We couldn’t spend more time on this problem so we just removed the category system from the search.
Has anybody successfully implemented a Verity search with categories? If you have, I really would like to hear from you.
November 6th, 2005 at 10:15 pm
Hi
Im having the same issue with just CF7 and verty in general. every time i call CFINDEX it takes about 10 seconds to index the doucment even when the collection is empty.
the doucments are only a few KB ..less than 100. When i run the same code on CF6.1 it will index in under 1 second.
This is even when i use collections with the category feature turned off.
#2: Martin Says:November 7th, 2005 at 12:51 am
Pat: I haven’t noticed any problems with indexing on CF7. The performance problems I mention in my post are to do with searching the collection.
#3: Richard Says:February 22nd, 2006 at 10:04 pm
Any updates on this front?
We’ve ran into the same problem (collection searches taking ~3 seconds), but we’ve only used verity since upgrading to CF7 and therefore have only setup verity with catagories. We never knew it could be faster, and have shyed away from using it because of the time issue.
We’re starting a new mach-ii site which will likely use verity. I’ll let you know if we make any progress.
#4: Martin Says:February 23rd, 2006 at 12:02 am
Richard: we just stopped using categories and performance went back to normal. Turns out we didn’t really need the category feature. We’ve been very happy with Verity since then. Good luck with your project!
#5: Alan Says:March 31st, 2006 at 8:27 pm
Any further info on this issue?
The categories feature seems really nice, but the Macromedia documentation is sparse.
#6: Martin Says:March 31st, 2006 at 8:31 pm
Alan: as I said in my previous comment, we gave up on the Verity categories concept. We didn’t really need them so haven’t spent any time and effort trying to make them work faster. Sorry…
#7: Blaine Says:October 25th, 2006 at 5:23 pm
Although the category concept is an excellent one, I have had continued frustration in getting it to work consistently for over a year now. Every once in awhile, for no apparent reason, reindexing the collection seems to kill the categories, although the cfcollection tag does not always report this correctly. Instead, the search simply returns no results when limited by category.
After countless hours of grief, Adobe finally acknowledged the bug and is “working on it”. Great.
#8: Jake Says:January 26th, 2007 at 3:07 am
One solution from Adobe was to clear the workspace http://www.adobe.com/go/cfe19ecb
Every time I do a cfindex, I also restart Verity K2, using a CFEXECUTE of the below batch file (*.bat). It only takes a second to restart.
net stop “ColdFusion MX 7 Search Server”
net start “ColdFusion MX 7 Search Server”
It is a hack but it solved the ‘lost’ categories problem.
#9: Jen Says:November 26th, 2007 at 11:14 am
The categories setting does indeed slow everything down! It was a double-whammy for me, since you can’t search multiple collections when you’re using categorylist, so I had to break it up into 4 tags with categorylist searches. It took the search from a quick half second (for around 60,000 records) to almost 10 seconds. (!!!)
I needed to use the categorylists, so taking the functionality out was not an option. But it was quite easy to just run a normal WITHOUT the categorylist attribute, and then just run a qoq to filter by categorylist…
select *
from v2_search
where categorytree LIKE ‘#currentsitetree#%’
order by category, score desc
I thought this seemed unneccessary - but since it dropped the speed of the entire search back down to less than half a second, it works for me! It also meant I could put the 4 collections back into one tag. Hurrah!
The categories stuff with verity is cool, but it still needs *a lot* of work.