Skip to main content


Showing posts from November, 2013

How to create hive external table for nutch's hbase webpage schema?

In order to query hbase table using hive, an external table should be created.
CREATE EXTERNAL TABLE webpage_hive (key string, baseUrl string, status int, prevFetchTime bigint, fetchTime bigint, fetchInterval bigint, retriesSinceFetch int, reprUrl string, content string, contentType string, protocolStatus string, modifiedTime bigint, prevModifiedTime bigint, batchId string, title string, text string, parseStatus int, signature string, prevSignature string, score int, headers map<string,string>, inlinks map<string,string>, outlinks map<string,string>, metadata map<string,string>, markers map<string,string>) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,f:bas,f:st,f:pts#b,f:ts#b,f:fi#b,f:rsf,f:rpr,f:cnt,f:typ,f:prot,f:mod#b,f:pmod#b,f:bid,p:t,p:c,p:st,p:sig,p:psig,s:s,h:,il:,ol:,mtdt:,mk:") TBLPROPERTIES ("" = "webpage");

Howto disable IPv6 Stack Ubuntu Linux

Sometimes network infrastructures are misconfigured to use ipv6 stack. Then you want to disable it at all. For ubuntu distribution it is easy as changing sysctl.conf and reboot your machine. Here is how to do it.

First check whether it is enabled or not:
$ cat /proc/sys/net/ipv6/conf/all/disable_ipv6
if output is 0 it is enabled and 1 it is disabled.

Adding the following rows is enough to disable ipv6 stack then restart your computer to take effect:
# vi /etc/sysctl.conf
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.all.disable_ipv6 = 1
# shutdown -r now

When your computer is up again, you can check it:
$ cat /proc/sys/net/ipv6/conf/all/disable_ipv6