Xavier Llorà » distributed computing

Parallel and Distributed Computational Intelligence book is out for pre-order

Xavier — Tue, 14 Sep 2010 04:17:35 +0000

“Parallel and Distributed Computational Intelligence” edited by Francisco Fernández de Vega & Erick Cantú-Paz and published by Springer is out for pre-order. The first chapter “When Huge is Routine: Scaling Genetic Algorithms and Estimation of Distribution Algorithms via Data-Intensive Computing” of the book was written together with coauthors Abhishek Verma, Roy Campbell, and David E. Goldberg describing how data-intensive computing can help push the size of problems that GAs and EDAs can address. You may find the abstact of the book below.

Abstract:

The growing success of biologically inspired algorithms in solving large and complex problems has spawned many interesting areas of research. Over the years, one of the mainstays in bio-inspired research has been the exploitation of parallel and distributed environments to speedup computations and to enrich the algorithms. From the early days of research on bio-inspired algorithms, their inherently parallel nature was recognized and different parallelization approaches have been explored. Parallel algorithms promise reductions in execution time and open the door to solve increasingly larger problems. But parallel platforms also inspire new bio-inspired parallel algorithms that, while similar to their sequential counterparts, explore search spaces differently and offer improvements in solution quality.

The objective in editing this book was to assemble a sample of the best work in parallel and distributed biologically inspired algorithms. The editors invited researchers in different domains to submit their work. They aimed to include diverse topics to appeal to a wide audience. Some of the chapters summarize work that has been ongoing for several years, while others describe more recent exploratory work. Collectively, these works offer a global snapshot of the most recent efforts of bioinspired algorithms’ researchers aiming at profiting from parallel and distributed computer architectures—including GPUs, Clusters, Grids, volunteer computing and p2p networks as well as multi-core processors. This volume will be of value to a wide set of readers, including, but not limited to specialists in Bioinspired Algorithms, Parallel and Distributed Computing, as well as computer science students trying to figure out new paths towards the future of computational intelligence.

Meandre: Semantic-Driven Data-Intensive Flows in the Clouds

Xavier — Sat, 15 Nov 2008 23:34:02 +0000

by Llorà, X., Ács, B., Auvil, L., Capitanu, B., Welge, M.E., Goldberg, D.E. (2008).

This paper has been accepted at the 4th IEEE International Conference on e-Science. An early draft of the paper can be found as IlliGAL technical report 2008013. You can download the pdf here. More information is also available at the Meandre website as part of the SEASR project.

Abstract:

Data-intensive flow computing allows efficient processing of large volumes of data otherwise unapproachable. This paper introduces a new semantic-driven data-intensive flow infrastructure which: (1) provides a robust and transparent scalable solution from a laptop to large-scale clusters,(2) creates an unified solution for batch and interactive tasks in high-performance computing environments, and (3) encourages reusing and sharing components. Banking on virtualization and cloud computing techniques the Meandre infrastructure is able to create and dispose Meandre clusters on demand, being transparent to the final user. This paper also presents a prototype of such clustered infrastructure and some results obtained using it.

ICEIS 2008: Blogging summary and final strings

Xavier — Wed, 18 Jun 2008 15:28:37 +0000

If you are looking for a list of the related blogging done during ICEIS 2008 just follow this link. During Sunday morning I run into Angel A. Juan, an assistant professor at Open University of Catalonia (UOC), interested on analyzing online teaching efforts and how tools to assist professors monitoring students performance on online media. I visited him yesterday at his office and we got and interesting exchange of ideas. Most of them revolved around the work we have conducted under the DISCUS project, and how similar is our efforts on marketing focus groups and their online teaching environment. His group, Distributed, Parallel and Collaborative Systems, was also interested on the work done under the SEASR project, mostly focusing on the Meadre infrastructure for data-intensive flow computing we are getting close to release.

ICEIS 2008: Final sprint and Ricardo Baeza-Yates

Xavier — Mon, 16 Jun 2008 11:10:46 +0000

This is the final sprint for ICEIS. I have been mostly focusing on posters this morning. It his hard to pick one up. I would just say that there was some interesting work on personalized recommender systems—paper 219. But as I said, there were a bunch of interesting ones and quite a few interesting by-the-poster conversations. Actually, I am having a very interesting time after the mix of attendees’ profiles.

The morning finally meandered into Ricardo Baeza-Yates‘s keynote talk. After the initial technical problems—presentation mode of OpenOffice running on Ubuntu 8.0.4 got up just 75% of the slide area—they finally succeeded on getting something up and get the talk started. This was a pretty technical talk about Yahoo! research effort on caching to improve the performance and also help scalability and contain cost on the coming years. Besides several cache techniques, he also presented a bunch of possible paralyzation models based on document/term partitions. A thing he breezed over was the machine learning model for classifying queries. That surfaced several places, from predicting common and rare content, to frequent, unfrequent, and rare queries. I was glad that the technical problems were solve and we could enjoy it.

And the conference is finally close. Next year, Milan.

ICEIS 2008: Blogging from Barcelona (Friday Morning)

Xavier — Fri, 13 Jun 2008 14:26:52 +0000

The hotel finally got the wireless system rebooted, so I am finally back up. I found quite unusual to open the ICEIS 2008 conference with a panel form by the four invited speakers: Moira Norrie (Global Information System Group, ETH), Jorge Cardoso (SAP), Jean-Marie Favre (UFR, IMA), Ricardo Baeza-Yates (Yahoo! Research Barcelona). Each of them did a short introduction of their invited talks, and just three questions were asked. Since the conference started at 11am, after the panel, lunch break

ZooKeeper and distributed applications

Xavier — Thu, 22 May 2008 13:39:36 +0000

Lately I have been exploring different alternatives for coordinating the execution of distributed applications. Yes, you guessed it right, I am working on the distribution of the execution of Meandre flows. Chopping the data-intensive flow and mapping the chunks onto a set of distributed processors requires several elements (graph analysis, resource management, etc.). However, the basic element that needs to be solved first is the need for a reliable and scalable coordination system.

During my trip to the Hadoop Summit and the Big Data Computing Study Group I ran into the ZooKeeper project, a Yahoo Research project. In their own words:

ZooKeeper is a high available and reliable coordination system. Distributed applications use ZooKeeper to store and mediate updates key configuration information. ZooKeeper can be used for leader election, group membership, configuration maintenance, etc.

There is also an interesting introductory lesson here, and some recipes for the most common data structures (queues, priority queues, distributed locks, etc.) are also available here. Looks promising, and may make the coordination of the distributed execution of a Meandre flows easier. The other reason that pushed me to explore this directions was that one of the Hadoop Summit highlights was the adoption of the ZooKeeper project by Hadoop itself.