[auton-users] Hadoop cluster

[Donghan (Jarod) Wang]

Dear Auton users,

I'm pleased to announce the availability of Auton cloud computing service,
a Hadoop cluster. The cluster consists of five nodes:

   - gd1
   - compute-0-0
   - compute-0-1
   - compute-0-2
   - compute-0-3

HDFS
=====
Your home directory is at /user/<your_auton_account> in the HDFS (31TB
available). There is no backup for the time being.

Tutorial
======
To run mapreduce jobs on the cluster, ssh to any of the above nodes.

Here is a short hadoop tutorial:

http://www.cloudera.com/content/cloudera-content/cloudera-docs/HadoopTutorial/CDH4/Hadoop-Tutorial.html

You'll need to forward corresponding port to your local compute to view
monitoring sites. Auton desktop users(Linux) can view them directly.

Monitoring sites
============

   - Job Tracker: http://gd1.int.autonlab.org:50030/jobtracker.jsp
   - HDFS Namenode: http://gd1.int.autonlab.org:50070/dfshealth.jsp

Services
=======

At this point, the cluster provides two services:

   - HDFS
   - MapReduce

Other services such as HBase, Hue will be added soon.

Best,

Jarod
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.srv.cs.cmu.edu/mailman/private/autonlab-users/attachments/20130628/b7626aab/attachment.html>