Facebook apps on App Engine without any third party API

12/17/2010 UPDATE


A lot of people found this blog posting because as we go through the authorization process, our users end up getting a "go to facebook.com" logo presented to their users when they first give their users a link to click on to authorize. This is happening because you need to put a target="_top" in the link.  I kid you not, if the user clicks on a link in your welcome page and they get a "Go to Facebook.com" message before they can go to the authorization screen, it's because you need to set target=_top in the link.  




It was a frustrating week or two while I tried to integrate Facebook into the google app engine. My goal was to use App Engine as my back end and Facebook as my front end for my apps.  I wanted users to add my app to their Facebook profile and grab a non-expiring oauth token for that user so I could access their Facebook data even when they were not using my app.  


The most important thing to know about doing this today is that any of  the third party apis like FaceBook4J jars are all completely useless here. Facebook exposes everything about a user using the graph api that can be accessed using http requests and getting json data back.  Trying to use these with app engine and the Google web toolkit will end up having you run into problems with libraries not being supported on either the server side or gwt client.  You don't need em - understanding oauth and urlfetch on app engine is all you need.


This blog posting covers the process of loading up your app in a Facebook canvas, getting the OAUTH token for your user, and then interacting with the Facebook graph API to get JSON data back from facebook for your user. All without any third party apis, SDK or anything else. I'll take you all the way to getting the current user's email. 


The only major assumption i'm making on this posting is that you are someone who is developing Java App Engine programs in the same way as the Google Stock Watcher tutorial 
http://code.google.com/webtoolkit/doc/1.6/tutorial/gettingstarted.html


So, you are familiar with the client / server namespaces and calling servlets from client interfaces to server side implementations. 

The first thing you need to do is set up a developer account and APP and on Facebook. Go to http://www.facebook.com/developers/ and set up a new application. You're going to need your app secret, api key and app ID.


Now, take the time to read over this manual (SERIOUSLY!) http://developers.facebook.com/docs/api it explains the Facebook standards for the graph api


Then go to app engine and create your corresponding app engine app. https://appengine.google.com/


I'm going to assume you also have eclipse IDE with the app engine add on and are able to create a new web application project and deploy it. 


Just for fun, I made an app on Facebook called Lucid Bubble, it's a simple dream journaling program that lets you record your dreams, tag your friends who are in the dream and post to the news feed about it. Not an app i would use every day but there seemed to be a demand for it. It also used app engine's mail hander so you can email a dream using your phone at bedside and post that way. 


Here is a screen shot of the end result: A nice app running inside a facebook iframe and using GWT, Google Charts and Facebook graph api data. The url for this app is http://apps.facebook.com/lucidbubble and it's being pulled from http://lucidbubble.appspot.com

Facebook App Settings:

Back on the Facebook app setting page. You can temporarily set the canvas URL to http://127.0.0.1:8888/. Later, you'll set that to your your deployed url on appspot.

Code:

I'm going to walk through the code that is first executed on the app engine entry point - OnModuleLoad() and the process of getting a code parameter, passing that code to the server side, doing oauth to get a user token and then requesting from facebook user data. I'm going to stop abruptly there because at that point you have everything you need to take to facebook from app engine.

When Facebook loads your app for the first time, it is the raw URL http://apps.facebook.com/yourapp. The first challenge is getting the code url parameter. 

Annoying Undocumented Facebook Behavior #1:

What is extremely annoying about this is that the Facebook docs tell you that to get the code you need to "Redirect the user to the authorization URL" but you can't redirect users in the iframe! If you do, you get an annoying and uninformative Facebook error page that says: "Go to Facebook.com" and when you click it, you actually go on to the authorization URL.  You also need to make sure you set the target=_top in your link

Ok, enough ranting, here is the client code: To break this down,  onModuleLoad() checks to see if the code is in the url parameter. If it is, we send the code to the server side for oauth processing (posted further down). If we do not have the code, we need to present the user with a welcome screen that has a link they can click on to go to add the app or authorize it. You can't just redirect the user to that url which would be nice.

private void loadWelcome()

{

//3: build a welcome page with a link to the authorise url

 

ContentPanel panel = PanelLibrary.VPanel();

viewport = new Viewport();

viewport.setLayout(new BorderLayout());

viewport.setBorders(true);

Anchor h = new Anchor();

h.setText("Click here to open your dream journal");

h.setHref(Facebook.getLoginRedirectURL());

h.setTarget("_top");

h.setStyleName("main-anchor");

panel.add(h, new RowData(-1, -1, new Margins(5,10,0,0)));  

viewport.add(panel);

RootPanel.get("main").add(viewport);

}

public void onModuleLoad() {

//1: do we have a code in the url?

final FaceBookServiceAsync facebookService = GWT.create(FaceBookService.class);

String  code = Location.getParameter("code");

if ( code == null)

{

//2: no, we need to bounce off the authorise url

loadWelcome();

}

else

{

//4: we must be back from the authorizes url, we have a code in the url - pass the code to the servlet:

facebookService.facebookLogin( code,Facebook.getClientId(), Facebook.getLoginRedirectURL(), Facebook.getSecret()  ,  new AsyncCallback(){

@Override

public void onFailure(Throwable caught) {

//loadWelcome();

Window.alert(caught.getMessage());

}

@Override

public void onSuccess(String email) {

//Window.alert(email);

loadLayout(email.toLowerCase());

}

private void loadLayout(String email) {

//5: all is well, load my app UI

}

});

}

}

LoadLayout is of course where your app starts to build the user interface, now that you have access to the users graph.

Here is the class I use for getting my api key, secrets and authorisation url, which is the first step to doing oauth.  Remember:  The redirect_URI must end with a /  and match the uri in your Facebook connect setting on the facebook app setting page. Also,

Annoying Undocumented Facebook Behavior #2

The redirect URI has a scope parameter that specifies what permisions your app needs, If you change these, Facebook may return an oauth token later in an unexpected format. I recomend you keep them the way I post them here and tweak them after you get everything working. Change anything below at your own risk, pay attention to trailing / and other params that will break your app. Make sure redirect_uri matches your app settings.

public class Facebook {
    // get these from your FB Dev App
    private static final String api_key = "1088xxxxxxx774d5ce9";
    private static final String secret = "secretkeyhere";
    private static final String client_id = "1530xx000432613";  
    public static String getClientId() {
return client_id;
}
    private static final String redirect_uri = "http://apps.facebook.com/lucidbubble/";
    public static String getAPIKey() {
        return api_key;
    }
    public static String getSecret() {
        return secret;
    }
    public static String getLoginRedirectURL() {
        return "https://graph.facebook.com/oauth/authorize?client_id=" +
            client_id + "&display=page&redirect_uri=" +
            redirect_uri+"&scope=user_status,publish_stream,offline_access,email";
    }
    public static String getAuthURL(String authCode) {
        return "https://graph.facebook.com/oauth/access_token?canvas=1&fbconnect=0&client_id=" +
            client_id+"&redirect_uri=" +
            redirect_uri+"&client_secret="+secret+"&code="+authCode;
    }
}


So far:

1. Your app should load and preset the user with a link to the authorization URL.
2. They'll then be sent back to the same page
3. OnModuleLoad will fire again but this time we'll have the code.

Everything else happens on the server all the way up to storing the users permanent OAUTH token and getting their Facebook info. This is all happening in my FacebookServiceImpl.java class that is called asynchronously from the module above. I'm assuming you know how to do this because you have gone through the app engine stock watcher tutorial :-)

 Get The Oauth Token.

With the code in hand, this is all you need to do to get the oauth token for a user on the server:


public String getToken(String code, String ClientID, String redirectURL, String secret)
{
String encodedCode="";
String retStr;
try {
encodedCode = URLEncoder.encode(code, "UTF-8");
String u1 = "https://graph.facebook.com/oauth/access_token?" +
"client_id=" +  ClientID + "&" +
"redirect_uri=" + redirectURL + "&" +  
"client_secret=" + secret + "&" +
"type=user_agent&" +
"code=" + encodedCode;
retStr = fetchURL(u1);
} catch (UnsupportedEncodingException e) {
retStr = null;
}
return retStr;



}


Pass to this function your client id, the same redirect url and secret stored in the Facebook class above. The exact same parameters you used in the client code to get the code. This function is using a generic function I wrote for taking any url, posting it and getting the result. Here is that function fetchURL:


private String fetchURL(String u) 
{
String retStr = "";
try 
{
           URL url = new URL(u);    
           BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
           String line;
                while ((line = reader.readLine()) != null) {
            retStr += line;
           
           }
           reader.close();


       } catch (MalformedURLException e) {
           // ...
       } catch (IOException e) {
           // ...
       }
return retStr;

}



Annoying Undocumented Facebook Behavior #3

Facebook does not return the plain oauth token, rather it returns the token like this: access_token=jhjvk... with the oauth token after the equals sign. If you want to use the token, you have to make sure it is url encoded. You'll see in my code here a simple function to use the token in this format, but removing the access_token= part, encoding  it and then putting it back together.


private String urlEncodeToken(String unencodedToken)
{
  String[] s =unencodedToken.split("=");
String retStr = null;
try {
retStr = "access_token=" + URLEncoder.encode(s[1], "UTF-8");
} catch (UnsupportedEncodingException e) {
}
return retStr;

}



Here is the complete facebook login function being called from the client. You'll see i'm

1. Using the code to get the token (above)
2. Using the token to get the email from https://graph.facebook.com/me  (me represents the authorized user, so that's the URL you use to get the users currently on your app)
3. Downloading the JSON data using my fetchURL functon (above)

At this point you're good to go, you have the users data and can interact with the Facebook graph. I also:

4. Convert the JSON into an object using GSON (google json api). My FBUser class is just a simple class with properties that match the json (email, name strings with getters and setters)  and a no-param constructor)


public String facebookLogin( String code, String ClientID, String redirectURL, String secret) 
{
                 String meURL =   "https://graph.facebook.com/me";
String token = getToken(  code,   ClientID,   redirectURL,   secret);
String email = "";
   String jsonEmail = fetchURL(meURL + "?" +  urlEncodeToken(token)  + "&fields=email,name");
FBUser fbuser = gson.fromJson(jsonEmail, FBUser.class);
email = fbuser.getEmail();
  return email;

}



You can now post and get from any of the url's documented in the graph api docs. http://developers.facebook.com/docs/api 

Authenticated Posts to App Engine Servlets

I thought I would take a moment to further document a Java class I wrote for taking a google account and password and then using it to make an authenticated POST and GET to a servlet hosted on app engine.

The process should be very simple:
1. Create an instance of my GoogleAuthentication.java class with your appspot url and credentials.
2. Use that object to add the auth cookie to your http post and get requests.

The Source code to this class is part of Nimbits4J.jar that allows developers to record time series data to the Nimbits Server running on app engine.  Nimbits is an open source data logger that runs on App Engine - you can learn more about it on www.nimbits.com.

The Source Code to Nimbits4J which contains the client classes is all posted under the GNU license version 3 on Google Code:

http://code.google.com/p/nimbits4j/

It contains a class called GoogleAuthentication.java  that handled getting the token, extracting the cookie information, and so on.

To the code!

With GoogleAuthentication.java or Nimbits4J.jar added to your project, setup these strings


        private final static String gaeAppBaseUrl = "http://appid.appspot.com/";
private final static String gaeAppLoginUrl = gaeAppBaseUrl + "_ah/login";
private final static String googleLoginUrl = "https://www.google.com/accounts/ClientLogin";
private final static String service = "ah"; 


Create the singleton object, passing in the above params, the gmail account and password a very trusting user gave you like this:

        GoogleAuthentication G = GoogleAuthentication.getNewGoogleAuthentication();
        G.ConnectClean(googleLoginUrl, service,  gaeAppBaseUrl,   gaeAppLoginUrl,   email,   password);

In the Nimbits4J project there is also a DataClient.java class which contains methods for talking to the Nimbits Services, but they are simply passing the client object G and the path to the service on app engine, with the params.  The code in red is where the magic happens.  On the Server Side, any calls to:

com.google.appengine.api.users.UserService u = UserServiceFactory.getUserService();

will return a valid user in these requests.


public static String doGet(String serviceUrl, String params, GoogleAuthentication G)
throws IOException {
String retVal = "";


try {
  URL url = new URL(postUrl + "?" + params);  //for example http://appid.appspot.com/service/yourservletmapping?foo=bar


HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("GET");
 
if (G != null)
{
connection.addRequestProperty("Cookie",G.getAuthCookie().getName() + "=" +            G.getAuthCookie().getValue());
}
BufferedReader reader = new BufferedReader(new InputStreamReader(
connection.getInputStream()));
 
String line;


while ((line = reader.readLine()) != null) {
retVal += line;
}
reader.close();


} catch (MalformedURLException e) {
throw e;


} catch (IOException e) {
throw e;
}


return retVal;


}




On the Server Side, your app engine should have your servlets mapped in the web.xml. A good sample of a service running on nimbits is here:

https://code.google.com/p/nimbits-server/source/browse/trunk/NimbitsServer/src/com/nimbits/server/service/CurrentValueService.java

Arduino Ethernet To App Engine Direct

So: An Arduino Ethernet Shield on your network and a Google App Engine Web Service out on the cloud, Eh?

Over the next couple of days i'm going to be writing all about how the Nimbits Data Logger Service on app engine can talk to an Arduino Micro controller directly from Google using and relay data feeds to Twitter, Facebook, Android, Excel, etc.

This Blog posting is a simplified discussion for those of you who want to talk to your own projects.  The challenge is the networking limitations of an Arduino device - even with EthernetDNS you still need to do your http POST and GET starting with an IP address.  You can use EthernetDNS to get the IP address of App Engine or Google.com but there is no IP address that points directly to your program; it's on the cloud.

I did solve this problem at first by putting an Ubuntu Server on my LAN and creating a Reverse Proxy to my App Engine URL http://nimbits1.appspot.com - So I could just hardcode the IP of my server into the Arduino.  But I wanted to go directly to App Engine without another tier.

The solution was very simple. If you do an HTTP POST or GET to google.com or appspot.com but provide the HOST Header in your request, google's infrastructure will direct your request correctly from there.


The Following Arduino code will do a GET from a Nimbits web service on App Engine. To keep it simple, I hard coded the IP of google.com. You could use EthernetDNS to get the current IP of Google.  The GET request will be sent to the appspot.com url list in the HOST: yourapp.appspot.com section below.   This is based on the Ethernet Client Example.



#include


byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192, 168, 1, 178 }; // a valid IP on your LAN
byte server[] =  {72, 14, 204, 104}; // Google.com or Appspot.com's IP


Client client(server, 80);


void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
  
  delay(1000);
  
  Serial.println("connecting...");
  
  if (client.connect()) {
    Serial.println("connected");
//this is a call to nimbits for the current value of a data point, but it can be any URL on app engine 
client.println("GET /service/currentvalue?point=test&email=bsautner@gmail.com HTTP/1.1");
client.println("Host:nimbits1.appspot.com");  //here is your app engine url
client.println("Accept-Language:en-us,en;q=0.5");
client.println("Accept-Encoding:gzip,deflate");
client.println("Connection:close");
client.println("Cache-Control:max-age=0");
client.println();
  } else {
    Serial.println("connection failed");
  }
}


void loop()
{
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }
  
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    for(;;)
      ;
  }
}

Authenticating Android to App Engine using the phone's credentials

If this helps you, please consider visiting the adds on this blog which supports the time i'm spending putting this stuff together. I'd also be very happy to see you follow this blog.

User this code at your own risk, it is provided as is with no warranty.

With the latest release of my Free and Social Data Logger App for Android (learn more about that on www.nimbits.com)  I took advantage of Android 2.2 account sync features. My app uses Google App Engine for a back end service and Restlet (www.reslet.org) to provide web services hosted on app engine. On Android we need to make requests to App engine with the users Google account attached to the request so App Engine knows who they are.

Actually, the Nimbits Android App is Open Source - as are all of the interfaces to Nimbits Data Logger - all of the code for the Android App is here: https://code.google.com/p/nimbits/

Don't forget to add Google credentials to the emulator if you're using one to debug your code.

Permissions:

In your Android Manifest, you'll want to add

• GET_ACCOUNTS
• USE_CREDENTIALS
• READ_OWNER_DATA 

Authentication Code

I'm sorry if my code isn't as clean and re-factored as I normally like it (maybe if things stop changing every week i'll be able to clean thing up)  If you add this code to your app, you'll want to go through and remove any of the refrences to my app: Nimbits and replace them with your apps name and url.

The class for authenticating to Google is GoogleAuthentication.java and it can be downloaded from our source repository.

The  most important part of that class is the getToken() function that is actually asking the phone for the google account.

 AccountManager mgr = AccountManager.get(context); 
Account[] accts = mgr.getAccountsByType("com.google"); 
 Account acct = accts[0];
AccountManagerFuture accountManagerFuture = mgr.getAuthToken(acct, "ah", null, (Activity) context, null, null);


This class will get an oauth token from your app. You can read more about that here:
http://code.google.com/appengine/docs/java/oauth/overview.html

Making the Web Service Call

I put all of my actual web service calls in once class (it's a small app) called DataClient.Java 

This class uses the GoogleAuthentication Class to get a token, and do GETs and POSTs to restlet services on my app engine site.


Let's walk through one of the functions:




public static Double getCurrentValue(String pointName) throws IOException
{
Double retVal;
  //1: Init the Singleton Authentication class
GoogleAuthentication G;
G = GoogleAuthentication.getGoogleAuthentication();
  //2: build the request URL - this is a restlet servlet on my app engine app
URL url = new URL("http://app.nimbits.com/nimbits/Service/value?pointname=" +       URLEncoder.encode(pointName,"UTF-8"));
 //3: Add the auth cookie the authentication class constructed.
URLConnection conn = url.openConnection ();
conn.addRequestProperty("Cookie",G.getAuthCookie().getName() + "=" + G.getAuthCookie().getValue());
  //4: post the request and download the result, in my case the current value of a data point
conn.setDoOutput(true);
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
String result ="";

while ((line = rd.readLine()) != null) {
result +=line;
}
rd.close();
retVal = Double.valueOf(result);


return retVal;


}



So, an android app with the correct permissions should be able to create an instance of the GoogleAuth class, which will provoke the phone into prompting its owner to grant your app permission to use their google credentials, and then make authenticated requests to app engine as if they were logged into it. All without making the user re-enter their password.