Terry’s Worklog

LibraryFind 0.8.4 upcoming changes

At some point, I’ll likely move this to the LibraryFind blog.  I just realized that I couldn’t remember my login information to post to the blog — so, I’ll post here.

I’m not exactly sure if the UI changes will be made in 0.8.4 to incorporate the new spawning/pinging (I think that they will), but the next version of LibraryFind will include an extended set of API that will allow users to develop UI interfaces that allow searches to be done independent of the UI.  This was done using a spawn plugin (you can find it on rubyforge or linked in lf via svn:externals), the creation of a jobs queue, and changes in a lot of the backend server components to make this all work.  I’ve just checked into the 0.8.4-ping branch (and soon into trunk once I’ve finished testing and create some unit tests) all the code necessary to make this work.  This requires the following changes to the LF codebase:

1. meta_search.rb refactor:  In order for this to work, I’ve had to move the cache checking code out of this file and move it into a more abstracted “connector” class.  The connector classes needed to be modified to support the new pinging functionality

2. New models: job_query.rb, job_item.rb, cache_search_session.rb, cached_search.rb, oai_search_class.rb, z3950_search_class.rb, opensearch_search_class.rb

3. API file changes: query_api.rb

4. Controller changes: meta_search.rb, query_conntroller.rb, dispatch.rb, record_set.rb

5. Migration changes: New migration has been added — 025_create_job_queues.rb

6. Changes to the environment.rb file (these will be in the environment.rb.example file)

New SOAP API elements:

• GetJobRecord (Queries the result set for a single job, returns the array of records)
• GetJobsRecords(Queries the record sets for an array of jobs, returns the array of records)
• SearchAsync (Returns an array of job ids, works like search)
• SimpleSearchAsync (Returns an array of job ids, works like simple search)
• CheckJobStatus (Returns a JobItem object for a single job)
• CheckJobsStatus (Returns an array of JobItem objects for an array of jobs)

The presumed workflow with the api is as follows:

1. Client will utilize the SearchAsync (or SimpleSearchAsync) API to initiate the search.  This will return an array of job ids.
2. Client will periodically ping the server, using CheckJobStatus (or CheckJobsStatus) for a list of jobs and their status.  Status is an enumeration.  -1 == error, 0 == finished/idle, 1 == searching
3. Client will retrieve records once items have been processed by calling GetJobRecord (or GetJobsRecords) for the array of records.

With this finished, I’ve completed pretty much all the current tags waiting for 0.8.4 and will be focusing on one additional enhancement that I’d like to see in 0.8.4 (if we can finish and test) or 0.8.5 — that being the addition of a REST-based XML api as well.  While I actually prefer the SOAP stack for some development, I would like to provide a REST-based api for more lightweight clients.

Anyway — that’s the news for now.

–TR

1 Comment to LibraryFind 0.8.4 upcoming changes

1. 

   Nice, glad to see the spawn plugin seems to be working for you. You ended up using it in fork rather than thread mode, I figure?
Jonathan Rochkind on November 21st, 2007

Leave a comment