Thursday, December 23, 2004
Ultra Recall
Kinook software have a new product called Ultra Recall, which in their words is "a personal information / knowledge / document management application for Microsoft Windows. It helps you capture, organize, and recall all of your electronic documents and information across all the applications that you use."
As far as a quick look goes, this seems to be a fairly direct competitor for Onfolio. I note a few immediate problems:
The Onfolio team have picked up on my earlier post about the inability to backup data (noting I make a "very good point about our lack of support for incremental back-ups of collections") - lets hope they can reconsider their reliance on the "one monster file approach" to address this. (I know that Joe Cheng who designed this part of the app covered why they went with .CFS in the first place, but the analysis he gives does not consider whether you can do backups, and is also factually wrong in saying that Windows has a 260 character limit for file paths - that was probably true in the bad old days of DOS based file systems, but has not been the case on NTFS for many years now).
As far as a quick look goes, this seems to be a fairly direct competitor for Onfolio. I note a few immediate problems:
- capture of information is via drag and drop, cut and paste, or other active actions - whereas if it ran inside my web browser, I could capture data by a much simpler "capture this" action
- it trumpets the fact that it includes "an in built web browser" - that's the last thing I want - give me a tool that runs inside my current web browser
- like Onfolio, it shares the singularly poor design decision of storing all its data in its own single database file - which rules out incremental backup (since trivial changes still cause the whole database to need to be backed up), and more or less rules out full backups (since the database is too big to fit on any backup media)
The Onfolio team have picked up on my earlier post about the inability to backup data (noting I make a "very good point about our lack of support for incremental back-ups of collections") - lets hope they can reconsider their reliance on the "one monster file approach" to address this. (I know that Joe Cheng who designed this part of the app covered why they went with .CFS in the first place, but the analysis he gives does not consider whether you can do backups, and is also factually wrong in saying that Windows has a 260 character limit for file paths - that was probably true in the bad old days of DOS based file systems, but has not been the case on NTFS for many years now).
# posted by zmarties @ 12/23/2004 03:29:00 pm 
Comments:
<< Home
:: is also factually wrong in saying that Windows has a 260 character limit for file paths
While it is true that NTFS as a filesystem does support pathnames longer than 260 characters (by always prepending the absolute path with "\\?\"), it's difficult if not impossible to actually work with such files and directories in practice. Microsoft's .NET libraries won't work with these superlong pathnames (this is the true gating factor for us), and I'd bet that large swaths of the Win32 API will fail as well. Furthermore, you can't use the shell to move/copy/delete such files, and basic command line utilities like "copy" will fail as well.
It actually would not be too big of a project to blow out the .CFS format into a directory-based format (i.e. using the same format, but mapping the structured storages/streams into filesystem directories/files), which would solve your backup problem--and probably improve write performance somewhat. (However, the actual contents of the directory would remain opaque as ever; the directory listing would just show you hundreds or thousands of small files that have GUIDs as names.)
I'm curious, do you actually have real collections that are larger than 650MB and 4.7GB? (You said "too big to backup to either CD or DVD".) If so, do you find Onfolio's performance to be acceptable under these conditions?
While it is true that NTFS as a filesystem does support pathnames longer than 260 characters (by always prepending the absolute path with "\\?\"), it's difficult if not impossible to actually work with such files and directories in practice. Microsoft's .NET libraries won't work with these superlong pathnames (this is the true gating factor for us), and I'd bet that large swaths of the Win32 API will fail as well. Furthermore, you can't use the shell to move/copy/delete such files, and basic command line utilities like "copy" will fail as well.
It actually would not be too big of a project to blow out the .CFS format into a directory-based format (i.e. using the same format, but mapping the structured storages/streams into filesystem directories/files), which would solve your backup problem--and probably improve write performance somewhat. (However, the actual contents of the directory would remain opaque as ever; the directory listing would just show you hundreds or thousands of small files that have GUIDs as names.)
I'm curious, do you actually have real collections that are larger than 650MB and 4.7GB? (You said "too big to backup to either CD or DVD".) If so, do you find Onfolio's performance to be acceptable under these conditions?
Just want to add that I am not at all above accepting criticism on the decision to implement collections as single files, or any other storage-related design decision. I made a decision based on how I thought users were going to use the product, what kinds of functionality were going to be important to them, and what kinds of functionality we were going to want to add down the road. And while many of those assumptions turned out to be right, many of those turned out to be wrong.
I'm not sure if I'd go so far as to abandon the single-file format--I feel it really has served most of our users very well--but I do recognize that there are many things we could've done better (and many other things we could've done worse, and gotten away with it).
I'm not sure if I'd go so far as to abandon the single-file format--I feel it really has served most of our users very well--but I do recognize that there are many things we could've done better (and many other things we could've done worse, and gotten away with it).
My collection grew to about 100MB in a couple of days - at which point I became seriously worried about the inability to do an incremental backup, so rethought my strategy for such research. This is the largest collection I've done much work with, and speed seemed acceptable.
Your point on the poor support for paths longer than 260 characters from even such universal tools as the shell and copy commands is well made. As you point out, without universal support for the long paths feature it can be very difficult to make practical use of it.
Your point on the poor support for paths longer than 260 characters from even such universal tools as the shell and copy commands is well made. As you point out, without universal support for the long paths feature it can be very difficult to make practical use of it.
Here's one solution for incremental backup of large files:
http://www.kinook.com/Forum/showthread.php?threadid=997
http://www.kinook.com/Forum/showthread.php?threadid=997
# posted by 
: 13/4/05 3:20 pm
: 13/4/05 3:20 pm
Trying to make that clickable: http://www.kinook.com/Forum/showthread.php?threadid=997
Post a Comment
# posted by : 13/4/05 3:21 pm
: 13/4/05 3:21 pm << Home
