Ippokratis Pandis

Compression has historically been used to reduce the cost of stor- age, I/Os from that storage, and buffer pool utilization, at the ex- pense of the CPU required to decompress data every time it is queried. However, significant additional CPU efficiencies can be achieved by deferring decompression as late in query processing as possible and performing query processing operations directly on the still-compressed data. In this paper, we investigate the benefits and challenges of performing joins… Show more

Compression has historically been used to reduce the cost of stor- age, I/Os from that storage, and buffer pool utilization, at the ex- pense of the CPU required to decompress data every time it is queried. However, significant additional CPU efficiencies can be achieved by deferring decompression as late in query processing as possible and performing query processing operations directly on the still-compressed data. In this paper, we investigate the benefits and challenges of performing joins on compressed (or encoded) data. We demonstrate the benefit of independently optimizing the compression scheme of each join column, even though join predi- cates relating values from multiple columns may require translation of the encoding of one join column into the encoding of the other. We also show the benefit of compressing “payload” data other than the join columns “on the fly,” to minimize the size of hash tables used in the join. By partitioning the domain of each column and defining separate dictionaries for each partition, we can achieve even better overall compression as well as increased flexibility in dealing with new values introduced by updates. Instead of decom- pressing both join columns participating in a join to resolve their different compression schemes, our system performs a light-weight mapping of only qualifying rows from one of the join columns to the encoding space of the other at run time. Consequently, join predicates can be applied directly on the compressed data. We call this procedure encoding translation. Two alternatives of encoding translation are developed and compared in the paper. We provide a comprehensive evaluation of these alternatives using product im- plementations of each on the TPC-H data set, and demonstrate that performing joins on encoded and partitioned data achieves both su- perior performance and excellent compression. Show less

Want to know how BLU Acceleration REALLY works? Written by the researchers and developers that invented it, this highly technical overview is not really a white paper, but in fact a contribution to academic literature that was refereed by world experts and presented at the 39th International Conference on Very Large Data Bases (VLDB 2013) in Riva del Garda, Trento, Italy on 27 August 2013.

The Blink project’s ambitious goal is to answer all Business Intelligence (BI) queries in mere seconds,
regardless of the database size, with an extremely low total cost of ownership...

The Blink project’s ambitious goals are to answer all Business Intelligence (BI) queries in mere seconds, regardless of the database size, with an extremely low total cost of ownership. It takes a very innovative and counter-intuitive approach to processing BI queries, one that exploits several disruptive hardware and software technology trends. Specifically, it is a new, workload-optimized DBMS aimed primarily at BI query processing, and exploits scale-out of commodity multi-core processors… Show more

The Blink project’s ambitious goals are to answer all Business Intelligence (BI) queries in mere seconds, regardless of the database size, with an extremely low total cost of ownership. It takes a very innovative and counter-intuitive approach to processing BI queries, one that exploits several disruptive hardware and software technology trends. Specifically, it is a new, workload-optimized DBMS aimed primarily at BI query processing, and exploits scale-out of commodity multi-core processors and cheap DRAM to retain a (copy of a) data mart completely in main memory. Additionally, it exploits proprietary compression technology and cache-conscious algorithms that reduce memory bandwidth consumption and allow most SQL query processing to be performed on the compressed data. Ignoring the general wisdom of the last three decades that the only way to scalably search large databases is with indexes, Blink always performs simple, “brute force” scans of the entire data mart in parallel on all nodes, without using any indexes or materialized views, and without any query optimizer to choose among them. The Blink technology has thus far been incorporated into two products: (1) an accelerator appliance product for DB2 for z/OS (on the “mainframe”), called the IBM Smart Analytics Optimizer for DB2 for z/OS, V1.1, which was generally available in November 2010; and (2) the Informix Warehouse Accelerator (IWA), a software-only version that was generally available in March 2011. We are now working on the next generation of Blink, called BLink Ultra, or BLU, which will significantly expand the “sweet spot” of Blink technology to much larger, disk-based warehouses and allow BLU to “own” the data, rather than copies of it. Show less

Middleware for ubiquitous and context-aware computing entails several challenges, including the need to balance between transparency and context-awareness and the requirement for a certain degree of autonomy. In this paper we outline most of these challenges, and highlight techniques for successfully confronting them. Accordingly, we present the design and implementation of a middleware infrastructure for ubiquitous computing services, which facilitates development of ubiquitous services… Show more

Middleware for ubiquitous and context-aware computing entails several challenges, including the need to balance between transparency and context-awareness and the requirement for a certain degree of autonomy. In this paper we outline most of these challenges, and highlight techniques for successfully confronting them. Accordingly, we present the design and implementation of a middleware infrastructure for ubiquitous computing services, which facilitates development of ubiquitous services, allowing the service developer to focus on the service logic rather than the middleware implementation. In particular, this infrastructure provides mechanisms for controlling
sensors and actuators, dynamically registering and invoking resources and infrastructure elements, as well as modeling of composite contextual information. A core characteristic of this infrastructure is that it can exploit numerous perceptual components for context acquisition. The introduced middleware architecture has been implemented as a distributed multi-agent system. The various agents have been augmented with fault tolerance capabilities. This middleware infrastructure has been exploited in implementing a non-obtrusive ubiquitous computing service. The latter service resembles an intelligent non-intrusive human assistant for conferences, meetings and presentations and is illustrated as a manifestation of the benefits of the introduced infrastructure.
Show less

Middleware for ubiquitous and context-aware computing entails several challenges, including the need to balance between transparency and context-awareness and the requirement for a certain degree of autonomy. In this paper we outline most of these challenges, and highlight techniques for successfully confronting them. Accordingly, we present the design and implementation of a middleware infrastructure for ubiquitous computing services, which facilitates development of ubiquitous services… Show more

Middleware for ubiquitous and context-aware computing entails several challenges, including the need to balance between transparency and context-awareness and the requirement for a certain degree of autonomy. In this paper we outline most of these challenges, and highlight techniques for successfully confronting them. Accordingly, we present the design and implementation of a middleware infrastructure for ubiquitous computing services, which facilitates development of ubiquitous services, allowing the service developer to focus on the service logic rather than the middleware implementation. In particular, this infrastructure provides mechanisms for controlling
sensors and actuators, dynamically registering and invoking resources and infrastructure elements, as well as modeling of composite contextual information. A core characteristic of this infrastructure is that it can exploit numerous perceptual components for context acquisition. The introduced middleware architecture has been implemented as a distributed multi-agent system. The various agents have been augmented with fault tolerance capabilities. This middleware infrastructure has been exploited in implementing a non-obtrusive ubiquitous computing service. The latter service resembles an intelligent non-intrusive human assistant for conferences, meetings and presentations and is illustrated as a manifestation of the benefits of the introduced infrastructure.
Show less