NOTE: If you are a developer, please use a private wiki based on foswiki/trunk on a daily base ...or use trunk.foswiki.org to view this page for some minimal testing.
Use Item11383 for general documentation changes for release 1.1.5. Use Item9693 for docu changes for release 2.0.
You are here: Foswiki>Tasks Web>Item9603 (02 Sep 2010, PaulHarvey) Edit this topic text (e) Attach an image or document to this topic; manage existing attachments (a) View sequential topic history View without formatting (v) Create new topic Printable version of this topic (p) More: delete or rename this topic; set parent topic; view and compare revisions (m)

Item9603: Creating archivist based on Cache::FastMmap

Priority: CurrentState: AppliesTo: Component: WaitingFor:
Normal Being Worked On Extension DBCacheContrib Main.PaulHarvey
This task is created out of a sudden, urgent need to get some new data online via Foswiki, and I'm running out of memory. Will be using CPAN:Cache::FastMmap

Don't worry, I won't be touching any existing code DBCache code, hopefully just a new archivist to choose from and a new script in tools/ to launch a fastmmap daemon that will look after the mmap cache file(s)

Goals:
  • One single ~1.3GB process with the mmap cache
  • Memory-efficiency for each fastcgi foswiki process (~200MB? Currently ~300MB, and then 1.1GB with the new data)
  • Faster than BDB; closer to Storable performance

... and from this I might have some lessons learnt to help make Foswiki's core search functionality more scalable too.

-- PaulHarvey - 02 Sep 2010

 

ItemTemplate edit

Summary Creating archivist based on Cache::FastMmap
ReportedBy PaulHarvey
Codebase
SVN Range
AppliesTo Extension
Component DBCacheContrib
Priority Normal
CurrentState Being Worked On
WaitingFor PaulHarvey
Checkins
TargetRelease n/a
ReleasedIn
Edit | Attach | Print version | History: r1 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r1 - 02 Sep 2010 - 07:37:19 - PaulHarvey
 
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. see CopyrightStatement. Creative Commons LicenseGet Foswiki at sourceforge.net. Fast, secure and Free Open Source software downloads