A Programming Framework for Crowd-Sensing Applications

Have you ever wished, in the middle of a research project, for a convenient way to recruit smartphone and tablet users worldwide to contribute data (images, videos, sound) using the sensors on their devices for your project?

Medusa is software that makes this capability, that we call crowd-sensing, possible. Medusa will let you program a crowd-sensing task easily and its software will automatically handle the tedious aspects of crowd-sensing: recruiting contributors, paying them, and processing and collecting data from mobile devices. Medusa also respects contributor privacy and anonymity.

The demo video explains Medusa with an example.

Medusa was developed by Moo-Ryong Ra in collaboration with Bin Liu, Tom La Porta, Ramesh Govindan, and Matthew McCartney at USC.

A thorough description of Medusa can be found at the Networked Systems Lab project page for Medusa

This document provides explains how to configure the Medusa cloud and client components.

This project is tested on the following platforms, devices, and dependencies:

• Medusa Cloud
  • OS: Ubuntu 9.04, 10.04, and Mint15
    • Basic dependencies
      • apt-get install python-mysqldb
      • apt-get install php5-common libapache2-mod-php5 php5-cli php5-curl
      • apt-get install sendmail-bin
  • TaskTracker
    • python 2.7.x
    • LAMP
      • Tested with Ubuntu 9.04(jaunty), 10.04(lucid), and Mint15
      • Apache web server (tested on 2.2.22)
      • Mysql (tested on 5.5.x)
      • PHP (tested on 5.4.9)
  • WorkerManager
    • apache-tomcat-6.0.35 with the jdk-1.5.x.
  • Websocket (GCM C2D alternative)
    • Requirements
      • PHP
        • PECL extension for PHP (aka, PEAR)
          • apt-get install php-pear
          • apt-get install php5-dev
          • apt-get install php5-mcrypt
      • ZeroMQ for communication between the Medusa Server and the WebSocket Server
        • Follow these instructions
          • Download the source, extract, and cd into it, the perform:
            • ./autogen, ./configure, make, and then make install
        • pecl install zmq-beta
        • add extension=zmq.so line to your to php.ini file
    • Optional Other Considerations
      • if you plan on connecting on the order of hundreds of Medusa clients, you may want to update your system’s maximum number of allowed per-process file descriptors
        • see ulimit for more details
      • Libevent speeds up server-side web socket operations
        • apt-get install libevent libevent-dev
        • pecl install libevent

        • add the extension=libevent.so line to your php CLI ini file (ussually in /etc/php5/cli/)

• Android client
  • Depends on the OpenCV-2.3.1 library
  • Tested on the following phones
    • Google Nexus 5 with Android 4.4.4
    • Google Nexus I with Android 2.3.6
    • Samsung Galaxy Nexus with Android 4.0.4

Let the root directory of the Medusa Cloud source code be medusa_cloud. Then make the medusa_cloud directory available to the web, so that the tasktracker can communicate with the client.

Prepare a linux account to access the mysql database for the Medusa server. Go to the medusa_cloud/config/ directory, and install the database schema.

mysql -u [mysql_username] -p[mysql password] < medusa_db_schema.sql

This creates a database named medusa and a set of tables inside of it. Login to mysql database, find CS_env table in the medusa database. Change the URLs appropriately. Specifically, the BASE-URL-DATA, SPC-HOST, SPC-URI, CS-DBHOST, and HIT-HOST parameters, which all relate to the web address and web location of the Medusa-cloud server.

• Update the following configuration files within the directory medusa_cloud/config/
  • db_host.info
    • The hostname on which the mysql server containing the medusa database is run
      • ex: localhost
  • db_account.info
    • The mysql_username|mysql_password of a mysql user with read and write permissions to the medusa database
      • ex: medusa_username|medusa_password
  • port.info
    • <medusa_acceptor_port>|<medusa_soc_port>|<web_socket_server_port>|<web_socket_zmq_port>
  • c2d_messaging_system.info
    • Set to either ws_c2dm (for websocket), c2dm (for GCM), or sms
  • remote_host.info
    • The public facing IP on which the mdscript_acceptor.py is run

Make sure you have a logs folder: mkdir medusa_cloud/tasktracker/logs

Install the Apache Tomcat server, then extract medusa_cloud/workermanager/medusa_hit_server.tar.gz to the apache-tomcat-root/webapps/medusa_hit_server/ directory. Verify that the CS_env table in the medusa database has the same information. Extract medusa_cloud/workermanager/config_files.tar.gz to /opt/config_files/.

You may be unable to store uploaded data or to run the programs generated by the MedAuthor web authoring tool. If this is the case, you may need to make the account that runs the medusa server and the account for the web server, e.g. www-data, in the same group. Then, go to the medusa_cloud/tasktracker/ and give write permissions for a set of directories as follows:

chmod g w data logs program

... and restart the apache web server.

The Medusa cloud originally was made up of two background processes, the runner and the acceptor. However, for the purposes of at least the MediaScope and testing the basic functinality of Medusa, only the acceptor is required. To start the process(es), change directories to medusa_cloud/tasktracker/ and run the following command(s) from within medusa_cloud/tasktracker/:

python ./mdscript_runner.py

python ./mdscript_acceptor.py

*Note that if you wish to run these processes in the background, you can simply supply the & keyword to the end and prepend the command with nohup to log all output to a file.

If using websockets and you get an error like this: Failed to bind the ZMQ: Address already in use Restart the server. If the issue still persists, change the ZMQ port used on line 13 of medusa_cloud/ws_server_php/bin/chat-server.php

Finally, if using GCM, you will need a Google GCM account. Register an account and modify the API_KEY stored in medusa_cloud/config/GCM.info

Now, the Medusa cloud is ready to run.

Open the G.java file in medusa_android/Medusa project. Change the following variables appropriately.

First, to use the TaskTracker properly, change the following entries.

/* Medusa cloud configuration */
public static final String SERVER_URL = "xxx.xxx.xxx.xxx";  /* Task tracker location */
public static final String URIBASE_UPLOAD = "http://"   SERVER_URL   "/Medusa/medusa-cloud/tasktracker/web_tt_upload.php";
public static final String URIBASE_REPORT = "http://"   SERVER_URL   "/Medusa/medusa-cloud/tasktracker/web_tt_service.php";

Second, you need a master account to use Google's GCM notification service.

//GCM
public static final String SENDER_ID = "GCM_SENDER_ID_HERE";

Third, to use reverse incentive mechanism as in the auditioning app, you should properly configure the following two values.

/* Worker's AMT Requestor ID */
public static final String AMT_WRID = "AKIAJ4DYABCDEFGHIJKL";
/* Worker's AMT Requestor Key. */
public static final String AMT_WRKEY = "oyg tsrPO3cQ85abcdefghijklmnopqrstuvwxyz";

Fourth, configure the websocket address port number

/* Web Socket Server*/
public static String WSS_PORT = "9002";

The first and the second are mandatory. The third and fourth are optional depending on your usage. Compile medusa_android/Medusa project and install it in a phone.

Note that if you wish build any additional medusalets, you must compile them and move the .apk file to medusa_android/Medusa/assets/

To verify all configurations are correct, execute a simple app.

Run Medusa client on the phone. Setup the client's WID using the menu or the automatic popup window, say wid.

On the server, go to medusa_cloud/tasktracker/tests/. Find hello_ws.xml, and set the wwid field the the wid you entered on the client.

To run the program, at medusa/cloud/tasktracker/tests/. execute the following command.

python ./run_xml.py program/hello_ws.xml

If you can see or here a HELLO WORLD message with voice instruction on the phone, the whole system is configured correctly.