話說阿~~參加完Hadoop in Taiwan 2012後回家有感而發, 立馬把之前"草稿"很久的東西給"發佈"出來! Hbase也是有table概念的, 以下就是今天的目標：
| table:scores | -->table name | |||||||
| Row\key | grade | course | -->column family | |||||
| english | music | math | history | art | chinese | -->column family qualifier | ||
| pablo | A | |||||||
| pablo | 90 | |||||||
| pablo | 0 | |||||||
| pablo | 70 | |||||||
| pablo | 60 | |||||||
| pablo | 50 | |||||||
| pablo | 40 | |||||||
利用Hbase shell打造出以上的table, 不要對terminal中的畫面有太多幻想, 以上的表是我自己用Calc畫出來的, terminal中就是像下圖一樣,
family , qualifier , row, key type....等等等的自己去找好兄弟google, 這些不在本篇的討論範圍, 因為blog都是筆記用, 不用來作學問XD
環境
跟這篇一樣
我的版本搭配是hadoop-1.0.3+hbase-0.94.1
PS:這篇沒有安裝zookeeper, 如果裝了zookeeper最下面的java對hbase做存取可能會出問題, 安裝可以參考這一篇
實作
[步驟一]
啟動大象, 啟動Hbase, 利用hbase shell指令進入Hbase操作畫面
等一下的table會叫做scores, 如果之前有自己建立過一樣名字的table請自己drop掉, 不drop掉的, 慢走不送了, drop方式請見下圖, 務必先disable再drop
[步驟二]
按照我Calc中設計的建立table, 給予兩個column family{'grade' , 'course'}, course還要給qualifier... (我懶得打字自己對照上面的表), grade只是描述等級而已不給qualifier
以下是簡單說明要用到的shell結構:
建立table的shell:
create 'table name' , 'column family1' , 'column family2' , 'column family3'
今天的講師說column family最好不要超過三個.......(筆記)
寫入資料的shell結構是(在hbase shell打錯會有很大心的提示給你):
put 'table name' , 'row' , 'column family:qualifier' , 'value'
接下來就直接key一大堆指令了
1.建立table
create 'scores','grade','course'
 |
put 'scores','pablo','grade','A'
 |
3.對course這一個column family底下的qualifier加入value
put 'scores','pablo','course:english','90'
put 'scores','pablo','course:music','0'
put 'scores','pablo','course:math','70'
put 'scores','pablo','course:history','60'
put 'scores','pablo','course:art','50'
put 'scores','pablo','course:chinese','40'
4.以上都輸入後,就用scan 'scores'來看看吧
眼睛尖的人是否有發現? 寫入Hbase後所有資料都排序了a-c-e-h-m-m, 沒錯~~Table會按Row key字典序排序如1,10,100,11,2,21,3......。
[步驟三]
以上shell操作就是那麼簡單, 接下來用java code來操作以上的shell指令
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
public class HBaseAnotherTest {
private static Configuration conf = null;
/*
* 初始化HBASE_CONFIG配置, 使每個method共用一樣的配置
*/
static {
Configuration HBASE_CONFIG = new Configuration();
/*
* 跟$HBASE_HOME/conf/hbase-site.xml中hbase.zookeeper.quorum屬性的值相同
*/
HBASE_CONFIG.set("hbase.zookeeper.quorum", "HDP141,HDP142,HDP143");
/*
* 跟$HBASE_HOME/conf/hbase-site.xml中hbase.zookeeper.property.clientPort屬性的值相同
*/
HBASE_CONFIG.set("hbase.zookeeper.property.clientPort", "2181");
conf = HBaseConfiguration.create(HBASE_CONFIG);
}
/**
* create table (create指令:create 'tableName','f2:q1','f1:q2','f2:q3'....)
*/
public static void creatTable(String tableName, String[] familys) throws Exception {
HBaseAdmin admin = new HBaseAdmin(conf);
if (admin.tableExists(tableName)) {
System.out.println("table already exists!");
} else {
HTableDescriptor tableDesc = new HTableDescriptor(tableName);
for(int i=0; i<familys.length; i++){
tableDesc.addFamily(new HColumnDescriptor(familys[i]));
}
admin.createTable(tableDesc);
System.out.println("create table " + tableName + " ok.");
}
}
/**
* delete table (先disable再drop)
*/
public static void deleteTable(String tableName) throws Exception {
try {
HBaseAdmin admin = new HBaseAdmin(conf);
admin.disableTable(tableName);
admin.deleteTable(tableName);
System.out.println("delete table " + tableName + " ok.");
} catch (MasterNotRunningException e) {
e.printStackTrace();
} catch (ZooKeeperConnectionException e) {
e.printStackTrace();
}
}
/**
* insert (put指令)
*/
public static void addRecord (String tableName, String rowKey, String family, String qualifier, String value) throws Exception{
try {
HTable table = new HTable(conf, tableName);
Put put = new Put(Bytes.toBytes(rowKey));
put.add(Bytes.toBytes(family),Bytes.toBytes(qualifier),Bytes.toBytes(value));
table.put(put);
System.out.println("insert recored " + rowKey + " to table " + tableName +" ok.");
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* delete one row
*/
public static void delRecord (String tableName, String rowKey) throws IOException{
HTable table = new HTable(conf, tableName);
List<Delete> list = new ArrayList<Delete>();
Delete del = new Delete(rowKey.getBytes());
list.add(del);
table.delete(list);
System.out.println("del recored " + rowKey + " ok.");
}
/**
* query one row (get指令:get 'tableName','column family:qualifier')
*/
public static void getOneRecord (String tableName, String rowKey) throws IOException{
HTable table = new HTable(conf, tableName);
Get get = new Get(rowKey.getBytes());
Result rs = table.get(get);
for(KeyValue kv : rs.raw()){
System.out.print(new String(kv.getRow()) + " " );
System.out.print(new String(kv.getFamily()) + ":" );
System.out.print(new String(kv.getQualifier()) + " " );
System.out.print(kv.getTimestamp() + " " );
System.out.println(new String(kv.getValue()));
}
}
/**
* list all (scan指令:scan 'tableName')
*/
public static void getAllRecord (String tableName) {
try{
HTable table = new HTable(conf, tableName);
Scan s = new Scan();
ResultScanner ss = table.getScanner(s);
for(Result r:ss){
for(KeyValue kv : r.raw()){
System.out.print(new String(kv.getRow()) + " ");
System.out.print(new String(kv.getFamily()) + ":");
System.out.print(new String(kv.getQualifier()) + " ");
System.out.print(kv.getTimestamp() + " ");
System.out.println(new String(kv.getValue()));
}
}
} catch (IOException e){
e.printStackTrace();
}
}
public static void main (String [] agrs) {
try {
String tablename = "scores";
String[] familys = {"grade", "course"};
HBaseAnotherTest.creatTable(tablename, familys);
/*
* add record pablo
*
*HBaseAnotherTest.addRecord("tablename","row","column family","column family qualifier","value");
*/
HBaseAnotherTest.addRecord(tablename,"pablo","grade","","A");
HBaseAnotherTest.addRecord(tablename,"pablo","course","english","90");
HBaseAnotherTest.addRecord(tablename,"pablo","course","music","0");
HBaseAnotherTest.addRecord(tablename,"pablo","course","math","70");
HBaseAnotherTest.addRecord(tablename,"pablo","course","history","60");
HBaseAnotherTest.addRecord(tablename,"pablo","course","art","50");
HBaseAnotherTest.addRecord(tablename,"pablo","course","chinese","40");
System.out.println("===========get one record========");
HBaseAnotherTest.getOneRecord(tablename, "pablo");
System.out.println("===========show all record========");
HBaseAnotherTest.getAllRecord(tablename);
// System.out.println("===========del one record========");
// HBaseAnotherTest.delRecord(tablename, "pablo");
// HBaseAnotherTest.getAllRecord(tablename);
System.out.println("===========show all record========");
HBaseAnotherTest.getAllRecord(tablename);
} catch (Exception e) {
e.printStackTrace();
}
}
}
**補充**
透過客戶端程式碼存取組態檔:
以上程式碼中使用的HBaseConfiguration類別提供兩種靜態方法
1.static Configuration create()
當呼叫此方法時背後的api會使用當前的classpath載入hbase-defuale.xml跟hbase-site.xml
2.static Configuration create(Configuration conf)
當呼叫此方法時指定一個現有個組態, 它的優先權會高於透過classpath所載入的設定檔
就把code貼到Eclipse中吧！run as hadoop前請先把之前用shell建立的table刪掉吧, 效果也是一樣的！
沒有留言:
張貼留言