Blog Archives

Temporary storage for Meandre’s distributed flow execution

Designing the distributed execution of a generic Meandre flow involves several moving pieces. One of those is the temporary storage required by the computing nodes (think of it as one node as one isolated component of a flow) to keep

Posted in Notes, Software Tagged with: , , , , , ,

Liquid: RDF endpoint for FluidDB

A while ago I wrote some thoughts about how to map RDF to and from FluidDB. There I explored how you could map RDF onto FluidDB, and how to get it back. That got me thinking about how to get

Posted in Notes, Software Tagged with: , , , , , ,

Liquid: RDF meandering in FluidDB

Meandre (NCSA pushed data-intensive computing infrastructure) relies on RDF to describe components, flows, locations and repositories. RDF has become the central piece that makes possible Meandre‘s flexibility and reusability. However, one piece still remains largely sketchy and still has no

Posted in Data-Intensive Computing, Notes, Research, Social Networks, Software Tagged with: , , , , ,

Efficient storage for Python

Did you ever run into the situation that your analysis/simulation data is too large to fit it in memory? Does the flat file format you use for storing your data sets become to big that renders it slow to a crawl? If

Posted in Notes Tagged with: , ,