Exposing services to CF applications

Service Broker API

The Service Broker (SB) API (full documentation) enables service providers to expose there offers to applications running on Cloud Foundry (CF). Implementing this contract, allows the CF Cloud Controller (CC) to communicate with the service provider in order to:

1. Catalog Management: register the offering catalog (e.g. different service plans), 
2. Provisioning: create/delete a service instance (e.g. create a new MongoDB collection),
3. Binding: connect/deconnect a CF application to a provisioned service instance.

For each of these possible actions, there an endpoint defined in the Service Broker contract.

1. Catalog Management
The Service Broker (full documentation) should expose an endpoint for catalog management that provides information on the service itself in a JSON format, the different plans (e.g. free or not) that can be consumed by applications, some meta-data that describe the service.

# The Cloud Controller sends the following request
GET http://broker-url/v2/catalog
# The Service Broker may reply as follows
< HTTP/1.1 200 OK
< Content-Type: application/json;charset=UTF-8
...
{

• services:
  

  [
  

  • {
    

    • planUpdatable: false,
    • id: "a unique service identifier",
    • name: "service name",
    • description: "service description",
    • bindable: true,
    • plan_updateable: false,
    • plans:
      

      [
      

      • {
        

        • id: "a unique plan id",
        • name: "plan name",
        • description: "plan description",
        • metadata: { },
        • free: false

        }

      ],
    • tags: [ ],
    • metadata: { },
    • requires: [ ],
    • dashboard_client: null

    }

  ]

}

2. Provisioning
The provisioning consists of synchronous actions that the Service Broker performs on demand from the CC to create a new or destroy an existing resource for the application. The CC sends PUT message with a designated instance identifier. Once the actions are performed, the Service Broker replies with the service and plan identifiers in a JSON format.

# The Cloud Controller sends the following request
PUT http://broker-url/v2/service_instances/:instance_id
{

• service_id: "service identifier"
  

  ,
• plan_id: "plan identifier"
  

  ,
• organization_guid: "ORG identifier"
  

  ,
• space_id: "SPACE identifier"
  

}
# The Service Broker may reply as follows
< HTTP/1.1 201 Created
< Content-Type: application/json;charset=UTF-8
...
{

• dashboard_url: null

}

A service instance once created can be updated (e.g. upgrading service consumption plan). For this, the same query is sent to the SB with a body containing only the attribute to update:
{

• plan_id: "new_plan_identifier"

}

3. Binding
Binding allows CF application to connect to a provisioned service instance and to start consuming the offered plan. When the SB receives a binding request from a CC, it replies with a the necessary information (e.g. service url, authentication information, etc.) for the CF application to utilize the offered service.

# The Cloud Controller sends the following request
PUT http://broker-url/v2/service_instances/:instance_id/service_bindings/:binding_id
{

• service_id: "service identifier"
  

  ,
• plan_id: "plan identifier"
  

  ,
• app_guid: "application identifier"

}
# The Service Broker may reply as follows
< HTTP/1.1 201 Created
< Content-Type: application/json;charset=UTF-8
...
{

• credentials:
  

  {
  

  • uri: "a uri to the service instance",
  • username: "username on the service",
  • password: "password for the username"

  }
• syslog_drain_url:
  
   null

}

For unbinding the application from the service, the SB receives on the same URL a request with a DELETE method.

Note! 
All previous requests from the Cloud Controller to the Service Broker contains the X-Broker-Api-Version HTTP header. It designates the Service Broker API (e.g. 2.4) supported by the Cloud Controller.

Managing Service Brokers

Once the previous endpoints are implemented, the SB can be registered to Cloud Foundry to be exposed to applications with the following command:
$ cf create-service-broker SERVICE_BROKER_NAME USERNAME PASSWORD http://broker-url/

To check if the service broker is successfully implemented
$ cf service-brokers

Other possible management operations are available to update, rename or delete a service borker
$ cf update-service-broker SERVICE_BROKER_NAME USERNAME PASSWORD http://broker-url/
$ cf rename-service-broker SERVICE_BROKER_NAME NEW_SERVICE_BROKER_NAME
$ cf delete-service-broker SERVICE_BROKER_NAME

Once the SB is created in CF database, its plans can be viewed with:
$ cf service-access

By default, the plans are all disabled, pick the service name from the output of the previous command and then:
$ cf enable-service-access SERVICE_NAME # enable access to service
$ cf marketplace -s SERVICE_NAME # output service plans

Managing Services

Once a service broker is available in the marketplace, an instance of the service can be created:
$ cf create-service SERVICE_NAME SERVICE_PLAN SERVICE_INSTANCE_NAME
Then service instances can be seen with:
$ cf services

Connecting service to application
To be able to connect an application to a service (running on a different network) and communicate with it, a route should be added through the definition of a Security group. Security groups allows you to control the outbound traffic of a CF app
$ cf create-security-group my_security_settings security.json

The content of security.json is as follows
[
  {
    "protocol": "tcp",
    "destination": "192.168.2.0/24",
    "ports":"80"
  }
]

Then, binding to a service instance should be performed as follows:
$ cf bind-service APP_NAME SERVICE_INSTANCE_NAME
Now, the application running on CF can access service instances through the credentials available from the environment variable VCAP_SERVICES.

Resources

• Managed services in CloudFoundry - link
• CloudFoundry and Apache Brooklyn for automating PaaS with a Service Broker - link
• Leveraging Riak with CloudFoundry - link

Pushing applications to CloudFoundry the Java way

CloudFondry provides a Java API that can be used to do anything just as the CLI. Follows are the steps that shows how to connect and publish an application to CF using Java code:

1. Skip SSL validation
You may have to skip SSL validation to avoid sun.security.validator.ValidatorException:

SSLContext ctx = SSLContext.getInstance("TLS");
X509TrustManager tm = new X509TrustManager() {
  public void checkClientTrusted(X509Certificate[] xcs, String string) {
  }
  public void checkServerTrusted(X509Certificate[] xcs, String string) {
  }
  public X509Certificate[] getAcceptedIssuers() {
    return null;
  }
};
ctx.init(null, new TrustManager[] { tm }, null);
SSLContext.setDefault(ctx);

2. Connect to CloudFoundry

Connect to the CloudFoundry API endpoint (e.g. https://api.run.pivotal.io) and authenticatewith your credentials:

String user = "admin";
String password = "admin";
String target = "https://api.10.244.0.34.xip.io";
CloudCredentials credentials = new CloudCredentials(user, password);
HttpProxyConfiguration proxy = new HttpProxyConfiguration("proxy_hostname", proxy_port);
CloudFoundryClientclient = new CloudFoundryClient(credentials, target, org, space, proxy);

3. Create an application

String appName = "my-app";
List urls = Arrays.asList("my-app.10.244.0.34.xip.io");
Staging staging = new Staging(null, "app_buildpack_git_repo");
client.createApplication(appName, staging, disk, mem, urls, Collections. emptyList());

4. Push the application

ZipFile file = new ZipFile(new File("path_to_app_archive_file"));
ApplicationArchive archive = new ZipApplicationArchive(file);
client.uploadApplication(appName, archive);

5. Check the application state

StartingInfo startingInfo = client.startApplication(appName);
System.out.println("Starting application: %s on %s", appName, startingInfo.getStagingFile());
CloudApplication application : client.getApplications()
System.out.printf("  %s (%s)%n", application.getName(), application.getState());

6. Disconnect from CloudFoundry

client.logout();

Sample Android OAuth client

OAuth is the Internet protocol that gives you an authorized access to resources on the Internet without grating user credentials. It has been deployed by many services like Twitter, Google, Yahoo or LinkedIn. And there are some libraries / code snippet in most every programming language that implement OAuth, but it is still hard to get it work (at least on Android).
Here is a sample code snippets based on the Signpost library that uses a custom WebView to intercept HTTP calls and handle them.
First the OAuthHelper.java the helper class:

private Context mContext;
private OAuthConsumer mConsumer;
private OAuthProvider mProvider;
private String mCallbackUrl;
private SharedPreferences mSettings;
private OAuthListener mListener;

public OAuthHelper(Context context, String consumerKey, String consumerSecret) {
   if ((consumerKey == null || "".equals(consumerKey)) && (consumerSecret == null || "".equals(consumerSecret))) {
      throw new IllegalArgumentException("You must specify your \"consumer Key\" and your \"consumer Secret\" when instantiating a OAuthHelper object");
   }
   mContext = context;
   mConsumer = new CommonsHttpOAuthConsumer(consumerKey, consumerSecret);
   mProvider = new CommonsHttpOAuthProvider(QypeConstants.REQUEST_TOKEN_URL, QypeConstants.ACCESS_TOKEN_URL, QypeConstants.AUTHORIZE_URL);
   mProvider.setOAuth10a(true);
   mCallbackUrl = QypeConstants.REDIRECT_URI;
     
   mSettings = context.getSharedPreferences(QypeConstants.PREFS_NAME, 0);
}

public void authorize(OAuthListener listener) 
   throws OAuthMessageSignerException, OAuthNotAuthorizedException, OAuthExpectationFailedException, OAuthCommunicationException {

   mListener = listener;
   OAuthLoginDialog oauthDialog = new OAuthLoginDialog(mContext, this);
   oauthDialog.loadUrl(getRequestToken());
   oauthDialog.show();
}

public String getRequestToken() 
   throws OAuthMessageSignerException, OAuthNotAuthorizedException,
   OAuthExpectationFailedException, OAuthCommunicationException {
   String authUrl = mProvider.retrieveRequestToken(mConsumer, mCallbackUrl);
   return authUrl;  
}

public String[] getVerifier(String uriString) {       
   return getVerifier(Uri.parse(uriString));
}

public String[] getVerifier(Uri uri) {
   // extract the token if it exists     
   if (uri == null) {
      return null;
   }
   String token = uri.getQueryParameter("oauth_token");
   String verifier = uri.getQueryParameter("oauth_verifier");
   return new String[] { token, verifier };
}

public String[] getAccessToken() {
   String access_token = mSettings.getString(ACCESS_TOKEN, null);
   String secret_token = mSettings.getString(SECRET_TOKEN, null);
   if(access_token==null || secret_token==null)
      return null;
   return new String[] {access_token, secret_token};
}

public void getAccessToken(final String verifier) {
   new Thread(new Runnable() {   
      public void run() {
         try {
            mProvider.retrieveAccessToken(mConsumer, verifier);
            mSettings.edit().putString(ACCESS_TOKEN, mConsumer.getToken()).commit();
            mSettings.edit().putString(SECRET_TOKEN, mConsumer.getTokenSecret()).commit();
            mListener.onOAuthComplete();     
         }catch(Exception e) {            
            e.printStackTrace();
         }
      }
   }).run();
}

public String request(String url) {
   String content = null;
   try {
       String accessToken[] = getAccessToken();
       mConsumer.setTokenWithSecret(accessToken[0], accessToken[1]);   
       HttpGet request = new HttpGet(url);
       // sign the request
       mConsumer.sign(request);
       // send the request
       HttpClient httpClient = new DefaultHttpClient();
       HttpResponse response = httpClient.execute(request);   
       content = EntityUtils.toString(response.getEntity());
   } catch (Exception e) {
       e.printStackTrace(); 
   }
   return content;
}

Second, the OAuthLoginDialog.java that will be used to intercept HTTP call back url

public class OAuthLoginDialog extends Dialog {
   private WebView mWebView;
   private OAuthHelper mHelper;

   public OAuthLoginDialog(Context context, OAuthHelper helper) {
      super(context);
      requestWindowFeature(Window.FEATURE_NO_TITLE);
      setContentView(R.layout.login_dialog);

      LayoutParams params = getWindow().getAttributes();
      params.height = LayoutParams.MATCH_PARENT;
      params.width = LayoutParams.MATCH_PARENT;
      getWindow().setAttributes((android.view.WindowManager.LayoutParams) params);

      mHelper = helper;

      init();
   }
   private void init() {
      mWebView = (WebView) findViewById(R.id.webView);
      mWebView.getSettings().setJavaScriptEnabled(true);

      mWebView.setWebViewClient(new WebViewClient() {
         @Override public void onReceivedError(WebView view, int errorCode, String description, String failingUrl) {}
         @Override public void onPageStarted(WebView view, String url, Bitmap favicon) {
             super.onPageStarted(view, url, favicon);
             progressBar.setVisibility(View.VISIBLE);
             if(url.startsWith(REDIRECT_URI)) {
                if(mHelper.getAccessToken() == null) {
                   String[] token = mHelper.getVerifier(url);
                   mHelper.getAccessToken(token[1]);
                }     
                mWebView.clearCache(true);
                dismiss();
             } 
         }

         @Override public void onPageFinished(WebView view, String url) {
             super.onPageFinished(view, url);
             progressBar.setVisibility(View.GONE);
         }

      });

   }
 
   public void loadUrl(String url) {
      mWebView.loadUrl(url);
   }
}

Third, the OAuth listener interface

public interface OAuthListener {
   // Called when the user is logged to the server
   public void onOAuthComplete();
   // Called when the user has canceled the login process
   public void onOAuthCancel();
   // Called when the login process has failed
   public void onOAuthError(int errorCode, String description, String failingUrl);
}

Finally, this is the code used to initiate an OAuth connection

OAuthHelper mHelper = new OAuthHelper(mContext, API_KEY, API_SECRET);
mHelper.authorize(new OAuthListener() {    
   public void onOAuthError(int errorCode, String description, String failingUrl) {}
   public void onOAuthComplete() {
      String testUrl = "http://sample_url_for_oauth_authorization/";
      mHelper.request(testUrl)
   }    
   public void onOAuthCancel() {}
});  

More resources can be found here implementing client side OAuth for android (it uses intent filter instead of a custom dialog) and here a sample OAuth for twitter.

Get the code for a sample project on Github.

Using Twitter Search API

Twitter Search API allows us to run searches against the real-time index of recent Tweets. 
Here is sample Python code showing how to search in twitter without using any Python third party library like twython, tweepy, python-twitter, tweetstream, etc.
Twitter search URL is http://search.twitter.com/search.format we will be using json as format for requested data.

import urllib
import json
query = "euro+2012" 
lat = str(37.781157)        
lng = str(-122.398720)
rd  = str(10000)
search = urllib.urlopen("http://search.twitter.com/search.json?geocode="+lat+","+lng+","+rd+"mi"+"&q="+query)
dict = json.loads(search.read())
for result in dict["results"]:         
    print result, "\n"
    print "*******************************" 

Printing result["text"] should give you the corresponding tweet message. To get the location from where the tweet was posted user result["location"].
You can get more information about the parameters (e.g. size of the returned result) to use in your search query here.