mardi 24 mars 2015

Password-less SSH root access

So I had to configure password-less SSH access between a master machine and a slave one:

1. Create an SSH key pair on the master machine
root@master-machine$ ssh-keygen 

2. Create an SSH key pair on the slave machine,
root@slave-machine$ ssh-keygen

To copy the public key to the remote machine we need a root access, however by default password-based SSH access as root is not allowed

3. On the slave machine: sudo passwd.
3.1. set a password for root (if not already set)
3.2. edit /etc/ssh/sshd_config (not /etc/ssh/ssh_config) to change PermitRootLogin without-password to PermitRootLogin yes.
3.3. restart SSH deamon with service ssh restart, if in an ssh session service ssh reload.

4. Copy master's root public key to the authorized keys in the slave machine
root@master-machine$ ssh-copy-id -i root@slave-machine

Disable password-based SSH access for root:
5. On the slave machine, edit /etc/ssh/sshd_config to change PermitRootLogin yes to PermitRootLogin without-password.

6. Now you can ssh as root from the master to the slave machine without password:
root@master-machine$ ssh root@slave-machine

For more details on SSH keys, check link.

vendredi 20 mars 2015

Exposing services to CF applications

Service Broker API

The Service Broker (SB) API (full documentation) enables service providers to expose there offers to applications running on Cloud Foundry (CF). Implementing this contract, allows the CF Cloud Controller (CC) to communicate with the service provider in order to:
  1. Catalog Management: register the offering catalog (e.g. different service plans), 
  2. Provisioning: create/delete a service instance (e.g. create a new MongoDB collection),
  3. Binding: connect/deconnect a CF application to a provisioned service instance.
For each of these possible actions, there an endpoint defined in the Service Broker contract.

1. Catalog Management
The Service Broker (full documentation) should expose an endpoint for catalog management that provides information on the service itself in a JSON format, the different plans (e.g. free or not) that can be consumed by applications, some meta-data that describe the service.

# The Cloud Controller sends the following request
GET http://broker-url/v2/catalog
# The Service Broker may reply as follows
< HTTP/1.1 200 OK
< Content-Type: application/json;charset=UTF-8
...
{
  • services:
    [
    • {
      • planUpdatable: false,
      • id: "a unique service identifier",
      • name: "service name",
      • description: "service description",
      • bindable: true,
      • plan_updateable: false,
      • plans:
        [
        • {
          • id: "a unique plan id",
          • name: "plan name",
          • description: "plan description",
          • metadata: { },
          • free: false
          }
        ],
      • tags: [ ],
      • metadata: { },
      • requires: [ ],
      • dashboard_client: null
      }
    ]
}

2. Provisioning
The provisioning consists of synchronous actions that the Service Broker performs on demand from the CC to create a new or destroy an existing resource for the application. The CC sends PUT message with a designated instance identifier. Once the actions are performed, the Service Broker replies with the service and plan identifiers in a JSON format.

# The Cloud Controller sends the following request
PUT http://broker-url/v2/service_instances/:instance_id
{
  • service_id: "service identifier"
    ,
  • plan_id: "plan identifier"
    ,
  • organization_guid: "ORG identifier"
    ,
  • space_id: "SPACE identifier"
}
# The Service Broker may reply as follows
< HTTP/1.1 201 Created
< Content-Type: application/json;charset=UTF-8
...
{
  • dashboard_url: null
}

A service instance once created can be updated (e.g. upgrading service consumption plan). For this, the same query is sent to the SB with a body containing only the attribute to update:
{
  • plan_id: "new_plan_identifier"
}

3. Binding
Binding allows CF application to connect to a provisioned service instance and to start consuming the offered plan. When the SB receives a binding request from a CC, it replies with a the necessary information (e.g. service url, authentication information, etc.) for the CF application to utilize the offered service.

# The Cloud Controller sends the following request
PUT http://broker-url/v2/service_instances/:instance_id/service_bindings/:binding_id
{
  • service_id: "service identifier"
    ,
  • plan_id: "plan identifier"
    ,
  • app_guid: "application identifier"
}
# The Service Broker may reply as follows
< HTTP/1.1 201 Created
< Content-Type: application/json;charset=UTF-8
...
{
  • credentials:
    {
    • uri: "a uri to the service instance",
    • username: "username on the service",
    • password: "password for the username"
    }
  • syslog_drain_url:
     null
}

For unbinding the application from the service, the SB receives on the same URL a request with a DELETE method.

Note! 
All previous requests from the Cloud Controller to the Service Broker contains the X-Broker-Api-Version HTTP header. It designates the Service Broker API (e.g. 2.4) supported by the Cloud Controller.

Managing Service Brokers

Once the previous endpoints are implemented, the SB can be registered to Cloud Foundry to be exposed to applications with the following command:
$ cf create-service-broker SERVICE_BROKER_NAME USERNAME PASSWORD http://broker-url/

To check if the service broker is successfully implemented
$ cf service-brokers

Other possible management operations are available to update, rename or delete a service borker
$ cf update-service-broker SERVICE_BROKER_NAME USERNAME PASSWORD http://broker-url/
$ cf rename-service-broker SERVICE_BROKER_NAME NEW_SERVICE_BROKER_NAME
$ cf delete-service-broker SERVICE_BROKER_NAME

Once the SB is created in CF database, its plans can be viewed with:
$ cf service-access

By default, the plans are all disabled, pick the service name from the output of the previous command and then:
$ cf enable-service-access SERVICE_NAME # enable access to service
$ cf marketplace -s SERVICE_NAME # output service plans

Managing Services
Once a service broker is available in the marketplace, an instance of the service can be created:
$ cf create-service SERVICE_NAME SERVICE_PLAN SERVICE_INSTANCE_NAME
Then service instances can be seen with:
$ cf services

Connecting service to application
To be able to connect an application to a service (running on a different network) and communicate with it, a route should be added through the definition of a Security group. Security groups allows you to control the outbound traffic of a CF app
$ cf create-security-group my_security_settings security.json

The content of security.json is as follows
[
  {
    "protocol": "tcp",
    "destination": "192.168.2.0/24",
    "ports":"80"
  }
]

Then, binding to a service instance should be performed as follows:
$ cf bind-service APP_NAME SERVICE_INSTANCE_NAME
Now, the application running on CF can access service instances through the credentials available from the environment variable VCAP_SERVICES.

Resources
  • Managed services in CloudFoundry - link
  • CloudFoundry and Apache Brooklyn for automating PaaS with a Service Broker - link
  • Leveraging Riak with CloudFoundry - link



mercredi 11 mars 2015

Pushing applications to CloudFoundry the Java way

CloudFondry provides a Java API that can be used to do anything just as the CLI. Follows are the steps that shows how to connect and publish an application to CF using Java code:

1. Skip SSL validation
You may have to skip SSL validation to avoid sun.security.validator.ValidatorException:
SSLContext ctx = SSLContext.getInstance("TLS");
X509TrustManager tm = new X509TrustManager() {
  public void checkClientTrusted(X509Certificate[] xcs, String string) {
  }
  public void checkServerTrusted(X509Certificate[] xcs, String string) {
  }
  public X509Certificate[] getAcceptedIssuers() {
    return null;
  }
};
ctx.init(null, new TrustManager[] { tm }, null);
SSLContext.setDefault(ctx);

2. Connect to CloudFoundry
Connect to the CloudFoundry API endpoint (e.g. https://api.run.pivotal.io) and authenticatewith your credentials:
String user = "admin";
String password = "admin";
String target = "https://api.10.244.0.34.xip.io";
CloudCredentials credentials = new CloudCredentials(user, password);
HttpProxyConfiguration proxy = new HttpProxyConfiguration("proxy_hostname", proxy_port);
CloudFoundryClientclient = new CloudFoundryClient(credentials, target, org, space, proxy);

3. Create an application
String appName = "my-app";
List urls = Arrays.asList("my-app.10.244.0.34.xip.io");
Staging staging = new Staging(null, "app_buildpack_git_repo");
client.createApplication(appName, staging, disk, mem, urls, Collections. emptyList());

4. Push the application
ZipFile file = new ZipFile(new File("path_to_app_archive_file"));
ApplicationArchive archive = new ZipApplicationArchive(file);
client.uploadApplication(appName, archive);

5. Check the application state
StartingInfo startingInfo = client.startApplication(appName);
System.out.println("Starting application: %s on %s", appName, startingInfo.getStagingFile());
CloudApplication application : client.getApplications()
System.out.printf("  %s (%s)%n", application.getName(), application.getState());

6. Disconnect from CloudFoundry
client.logout();

mardi 24 février 2015

Installing Cloud Foundry v2 locally on Vagrant

Cloud Foundry (CF)

CloundFoundry (CF) is one of the many PaaS available out there that aims to empower developers to build their applications (e.g. web) without caring about infrastructure details. The PaaS handles the deployment, scaling and management of the apps in the cloud data center, thus boosting the developer productivity.
CF has many advantages over other PaaS solutions as it is open source, it has a fast growing community and many big cloud actors are involved in the development and spreading it adoption. It also can be run anywhere even on a laptop and this what this post is about. So keep reading..

Terminology

- Bosh is an open-source platform that helps deploying/managing systems on cloud infrastructures (AWS, OpenStack/CloudStack, vSphere, vCloud, ect).
- Bosh Lite is a lightweight version of Bosh that can be used to deploy systems locally by using Vagrant instead of cloud infrastructure (e.g. AWS) and Linux Containers (Warden project) for to run your system instead of VMs.
- Stemcell is a template VM that will be used by Bosh to create VMs and deploy them to the cloud. I contains essentially an OS (e.g. CentOS) and a Bosh Agent in order to be controlled.

1. Install Git
$ sudo apt-get install git

2. Install VirtualBox
$ sudo echo "deb http://download.virtualbox.org/virtualbox/debian precise contrib" >> /etc/apt/sources.list
or create a new .list file as described in this thread.
$ wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install virtualbox-4.3
$ sudo apt-get install dkms
$ VBoxManage --version
4.3.10_Ubuntur93012

3. Install Vagrant (the known version to work with bosh-lite is 1.6.3 - link)
$ wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.3_x86_64.deb
$ sudo dpkg -i vagrant_1.6.3_x86_64.deb
$ vagrant --version
Vagrant 1.6.3

Check if vagrant is correctly working with the installed virtual box
$ vagrant init hashicorp/precise32
$ vagrant up

4. Install Ruby(using RVM) + RubyGems + Bundler
4.1. Install rvm
$ curl -sSL https://rvm.io/mpapis.asc | gpg --import -
$ curl -sSL https://get.rvm.io | bash -s stable
$ source /home/{username}/.rvm/scripts/rvm
$ rvm --version

4.2. Install latest ruby version
$ rvm install 1.9.3-p551
$ ruby -v
ruby 1.9.3p551 (2014-11-13 revision 48407) [x86_64-linux]

5. Install Bosh CLI (check the prerequisites for the target OS here)
- Note that Bosh CLI is not suppored on windows - github issue
$ sudo apt-get install build-essential libxml2-dev libsqlite3-dev libxslt1-dev libpq-dev libmysqlclient-dev
$ gem install bosh_cli

6. Install Bosh-Lite
$ git clone https://github.com/cloudfoundry/bosh-lite
$ cd bosh-lite
$ vagrant up --provider=virtualbox

In case the following message is seen The guest machine entered an invalid state while waiting for it to boot, then:
  • check if virtualisation (Intel VT-x / AMD-V for 32bits or Intel EPT / AMD RVI for 64bits) is enabled on target system here. If not then enable it from the BIOS, for ESXi check link1 and link2 and add vhv.enable = "TRUE" to the vm configuration file (i.e. vmx) and make sure the VM is of version 9. 
  • You may also have to check if USB 2.0 controller is enabled, if it is then disable it.
Target the BOSH Director
$ cd ..
$ bosh target 192.168.50.4 lite
$ bosh login
Your username: admin
Enter password: *****

Logged in as `admin'

Setup a route between the laptop and the VMs running inside Bosh Lite
$ cd bosh-lite
$ ./bin/add-route

7. Deploy Cloud Foundry
Install spiff
$ brew tap xoebus/homebrew-cloudfoundry

$ brew install spiff

$ spiff
To install spiff on linux systems check this issue.

Upload latest stemcell
$ wget http://bosh-jenkins-artifacts.s3.amazonaws.com/bosh-stemcell/warden/latest-bosh-stemcell-warden.tgz
$ bosh upload stemcell latest-bosh-stemcell-warden.tgz
Check the stemcells
$ bosh stemcells

Upload latest CF release
$ git clone https://github.com/cloudfoundry/cf-release
$ export CF_RELEASE_DIR=$PWD/cf-release/
$ bosh upload release cf-release/releases/cf-XXX.yml

Deploy CF releases
$ cd bosh-lite/
$ ./bin/provision_cf
$ bosh target # check the target director
$ bosh vms    # check the installed VMs on the cloud

Manually (to be continued)
Generate a configuration file manifests/cf-manifest.yml
$ mkdir -p go
$ export GOPATH=~/go
$ cd bosh-lite
$ ./bin/make_manifest_spiff

Deploy release
$ bosh deploy

Install CF CLI

Play with CF
$ cf api --skip-ssl-validation https://api.10.244.0.34.xip.io
$ cf login
$ cf create-org ORG_NAME
$ cf orgs
$ cf target -o ORG_NAME
$ cf create-space SPACE_NAME
$ cf target -o ORG_NAME -s SPACE_NAME

To access the VM from the LAN (i.e. another machine):
  1. Install an HTTP Proxy (e.g. squid3),
  2. Configure CF HTTP_PROXY environment variable, and 
  3. Configure the proxy:
       $ sudo nano /etc/squid3/squid.conf 
       acl local_network src 192.168.2.0/24
       http_access allow local_network

Stopping CF
Shooting down bosh-lite VM can be surprisingly tricky. May better stop the VM with:

  • vagrant suspend to save current state for next start up, or
  • vagrant halt, then next time to start CF use vagrant up followed by bosh cck (documentation).


Troubleshooting
$ bosh ssh # then choose the job to access (password: admin)
bosh_something@something:~$ sudo /var/vcap/bosh/bin/monit summary
Find the Bosh Lite IP address
$ cd bosh-lite/
$ vagrant ssh
vagrant@agent-id-bosh-0:~$ ifconfig
vagrant@agent-id-bosh-0:~$ exit

Complete installation script can be found here.

Resources
  • Installing latest versions for virtualbox and vagrant - link
  • Installing ruby with rvm - link.
  • DIY PaaS (CF v1) running DEA link1, stagging applications link2.
  • Deploying CF Playground (a kind of web admin interface) - link
  • Installing CF on vagrant - link video
  • Installing BOSH lite - github repo, tutorial
  • Deploying CF using BOSH lite - github repo, demo
  • http://altoros.github.io/2013/using-bosh-lite/
  • Installing a new hard drive - link
  • xip.io a free internet service providing DNS wildcard - link
  • Troubleshooting with Bosh CLI - official doc, app health, monit summary
  • Remotely debug a CF application - link
  • CloudFoundry manifest.yml generator - link


jeudi 4 septembre 2014

Getting started with Hive

Introducing Hive

Hive installation is straightforward (no much things to configure)
$ wget http://mir2.ovh.net/ftp.apache.org/dist/hive/stable/apache-hive--bin.tar.gz
$ tar xzf apache-hive--bin.tar.gz
$ cd apache-hive--bin/bin/
hive> show tables;

Notice that the environment variable HIVE_HOME is not required (which is not the case for hadoop/hbase/tez). Also, hive-site.xml is not required but if we want to use an hdfs directory then it should contain something like:
<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>hdfs://namenode_hostname/user/hive/warehouse</value>
  <description>location of default database for the warehouse</description>
</property>

Introducing HQL

So lets create a table 'quotes' and make it available to other hadoop programs as a text file:
hive> CREATE EXTERNAL TABLE quotes (symbol STRING, name STRING, price DOUBLE)
    > ROW FORMAT DELIMITED FIELDS TERMINATED by ',' LINES TERMINATED BY '\n'
    > STORED AS TEXTFILE
    > LOCATION '/tmp/quotes.txt';

Then, we can load data into these tables, for instance form a local file quotes.csv that looks like:
"GE","General Electric ",28.09
"MSFT","Microsoft Corpora",41.66
"GOOG","Google Inc.",604.83
"GM","General Motors Co",41.85
"FB","Facebook, Inc.",72.59
"AAPL","Apple Inc.",607.33
"T","AT&T Inc.",37.15
"VZ","Verizon Communica",52.06
"TM","Toyota Motor Corp",134.94

with the following query:
hive> LOAD DATA LOCAL INPATH '/path/to/quotes.csv'
    > OVERWRITE INTO TABLE quotes;

Once the table is filled, we can query it with things like:
hive> SELECT * FROM quotes;
hive> SELECT symbol FROM quotes;

We can export and save the result of a query into a file locally say /tmp/..:
hive> INSERT OVERWRITE LOCAL DIRECTORY '/tmp/quotes_100'
SELECT *
    > FROM quotes
    > WHERE quotes.price > 100;
The result of this export is a set of files under the quotes_100 directory, the list of quotes that match the criteria can be found in a file name 000000_0

Tuning Hive

Understanding the underlying details of how Hive plan when executing queries is essential for performance tuning. One way to understand the query plan is the use of the EXPLAIN key word:
hive> EXPLAIN SELECT * FROM quotes
    > FROM quotes
    > WHERE quotes.price > 100;
hive> EXPLAIN SELECT SUM(price) FROM quotes;
The result shows the translation of these queries into different possible operations called stages, for instance map-reduce, sampling, merge, or limit stages.
The use of the keyword EXTENDED along with explain will provide even further details for the query execution plan:
hive> EXPLAIN EXTENDED SELECT SUM(price) FROM quotes;

By default, hive executes a stage at once. This default behavior can be overridden by setting the property to true in hive-site.xml:
<property>
  <name>hive.exec.parallel</name>
  <value>true</value>
  <description>Whether to execute jobs in parallel</description>
</property>

The number of mappers/reducers launched is determined by the size of the input files divided by the default size attributed to a given task, it can be configured via:
<property>
  <name>hive.exec.reducers.bytes.per.reducer</name>
  <value>750000000</value>
  <description></description>
</property>

Resources

lundi 1 septembre 2014

Troubleshooting ubuntu server network interface

So I've installed Ubuntu server on VirtualBox and when I activated a second network adapter with a bridged mode, the later was not automatically configured on Ubuntu.
In fact, the interace cannot be seen with ifconfig and ifconfig -a showed it as disabled.
I tried to bring it up and restart networking service:
$ifconfig eth1 up
$/etc/init.d/networking restart
Now the interface is active but it has only an IPv6 address and when I restart the virtual machine, the interface goes disabled again.
When checking the /etc/network/interfaces there was no eth1!!, so I added it in order to be configured automatically:
$vi /etc/network/interfaces

auto eth1
iface eth1 inet dhcp

that's it now the interface works fine.

lundi 18 août 2014

Comparison between caching systems for Java


Servers are getting more and more powerful with a lot of RAM (up to hundred to thousands of giga bytes). However, it is still not possible to use most of the available capacity directly in java applications due to inherent limitations of the GC (Garbage Collector) on JVM that may pause the application for a long time (even up to many minutes) to move objects between different generations.

Follows is the description/comparison between some solutions, also called data grids like, that can be used to face this problem like the Infinispan project of JBoss (ex. JBoss Cache), DirectMemory (an Apache proposal), EhCache (of terracotta), etc.

Caches

1. Infinispan (JBoss Data Grid Platform)
  • Don't provide support for expiration events as disscussed in the forum.
  • SingleFileCacheStore a cache loader from a file stores that manages the data activation (loading from store to cache) and passivation (saving data to store).
  • List of possible attributes in the XML configuration for infinispan 4.0 and infinispan 6.0.

2. MapDB
  • Exists only in the embbeded mode
  • Enables the creation of on heap and off-heap collections (map, queue), as well as file-backed collections
  • Listeners registerd to cache events are notified in the main thread (i.e. should implement async notifications)
  • Can be used for lazy loading (e.g. Lazily_Loaded_Records.java).
  • Provides means for pumping the integral data available on memory to disk (e.g. Pump_InMemory_Import_Then_Save_To_Disk.java).
  • Transaction isolation level is Serializable which is the highest level and means a new transaction can be initiated only if previous one was committed. 
  • Transactions uses a global lock which reduce considerably the cache performance.

3. Akiban's Persistit - github
4. JCS (Java Caching System)
5. Hazelcast
6. GridGain



5. Others: LArray, Cache2K, DirectMemory (initial project on github, apache proposal for incubation) an off-heap memory storage, MVStore the storage subsystem of the H2 database, Spring cache, HugeCollections.

Search
Resources
  • A good explanation of the use of ByteByffer to build non-heap memory caches by Keith Gregory: blog post, JUG presentation, another one.
  • An article on InfoQ about HashMap implementation for off-heap map.
  • An ibm red book on capacity for big data and off-heap memory.
  • Examples related to the use of EhCache from a Devoxx 2014 presentation.
Benchmarks
  • Cache2K vs Infinispan/EhCache/JCS - bench
  • Radargun a framework for benchmarking data grids
Memory storage

In-memory databases (a detailed description can be found at Information Week):
  • NoSQL approaches (covers the class of nonrelational and horizontally scalable databases) like Aerospike.
  • NewSQL approaches (emerging databases offerting NoSQL scalability but with familiar SQL query capabilities, i.e. SQL-compliant) like VoltDB, Oracle TimesTen, IBM solidDB, MemSQL.
Companies like Microsoft, Oracle and IBM choosed to add the in-memory support for their traditional databases (e.g. moving tables to memory), whereas SAP adopted another approach with its Hana platform that aims to put everything in-memory.


Some traditional RDBMS can be configured to store their data in-memory instead of disk storage like sqlite, MySQL, etc.