Python REST API oAuth Problem

18 posts / 0 new

Joined: 10/25/2011

Sun, 02/05/2012 - 12:49

Hi all,

New member here. Please forgive me for my immaturity with Python; I have only just started using the language for a project at University so I might sound entirely ineducate!

I am attempting to use the python REST API to store my connections' data and analyse it using visualisation techniques. The problem is, I can't seem to be able to pass the oAuth dance correctly, possibly because not only have I not understood the details that need to be placed in my code, but also because I've never done it before!

My code is as follows:
# -*- coding: utf-8 -*-

import os
import sys
import webbrowser
import cPickle
from linkedin import linkedin

# Parses out oauth_verifier parameter from window.location.href and
# displays it for the user

RETURN_URL = 'http://miningthesocialweb.appspot.com/static/linkedin_oauth_helper.html'

def oauthDance(key, secret, return_url):
api = linkedin.LinkedIn('mykey', 'mysecret',' ', )

result = api.requestToken()
print result

if not result:
print >> sys.stderr, api.requestTokenError()
return None

authorize_url = api.getAuthorizeURL()

webbrowser.open(authorize_url)

oauth_verifier = raw_input('PIN number, bro: ')

result = api.accessToken(verifier=oauth_verifier)
if not result:
print >> sys.stderr, 'Error: %s\nAborting' % api.getRequestTokenError()
return None

return api

# First, do the oauth_dance

api = oauthDance('mykey', 'mysecret', ' ', )

# Now do something like get your connections:

if api:
connections = api.GetConnections()
else:
print >> sys.stderr, 'Failed to aunthenticate. You need to learn to dance'
sys.exit(1)

# Be careful - this type of API usage is "expensive".
# See http://developer.linkedin.com/docs/DOC-1112

print >> sys.stderr, 'Fetching extended connections...'

extended_connections = [api.GetProfile(member_id=c.id, url=None, fields=[
'first-name',
'last-name',
'current-status',
'educations',
'specialties',
'interests',
'honors',
'positions',
'industry',
'summary',
'location',
]) for c in connections]

# Store the data

if not os.path.isdir('out'):
os.mkdir('out')

f = open('out/linkedin_connections.pickle', 'wb')
cPickle.dump(extended_connections, f)
f.close()

print >> sys.stderr, 'Data pickled to out/linkedin_connections.pickle'

This is based on an excerpt from O' Reilly's 'mining the social web' so it should work. I have installed python-linkedin and I know it's a 3rd party plugin that isn't strictly supported, but if I can manage the login, I should be able to move ahead afterwards. Any halp would be fantastic!

I shall suck my thumbs till then.

Kirsten Jones

Joined: 06/30/2011

#2 Mon, 02/06/2012 - 07:59

No worries, OAuth is tricky stuff. We do have some python examples in our Quick Start Guide, and you can use that to get started - it uses the oauth2 library and you can start from there. Once you've done the dance you'll have a token and secret you can use for development.

Robert Schuon

Joined: 07/19/2011

#3 Mon, 02/06/2012 - 15:29

Well, Kirsten, I have experience with a bunch of API's, and I can't figure your docs out. For example, where do you get the call, make_request? Since you don't show any of the Python import statements, it makes it very hard to see what libraries need to be loaded. I have so far spent 6 hours trying to get one share request working, reading your docs, StackOverflow, anything I can. Since make_request doesn't seem to be part of OAuth2, I can't use it.

So here is sample code:

share_url = 'http://api.linkedin.com/v1/people/~/shares'
consumer = oauth.Consumer(settings.LINKED_IN_API_KEY, settings.LINKED_IN_API_SECRET)
client_token = oauth.Token(self.oauth_token, self.oauth_token_secret)
client = oauth.Client(consumer, client_token)
json_content = json.dumps(share_object)
headers = {"Content-Type":"application/json"}
response = client.request(share_url,"POST",json_content, headers=headers)

Any ideas? Thank you in advance.
Bob

Kirsten Jones

Joined: 06/30/2011

#4 Mon, 02/06/2012 - 16:39

If you download the Source Code Tarball as suggested in getting started, it has all of the code you need.

But here's a full sample:

import oauth2 as oauth
import time
import simplejson
 
url = "http://api.linkedin.com/v1/people/~/shares"
 
consumer = oauth.Consumer(
        key="xxx",
        secret="xxx")
        
token = oauth.Token(
        key="xxx", 
        secret="xxx")
 
 
client = oauth.Client(consumer, token)
body = {"comment":"Posting from the API using JSON",
                "content":{
                        "submitted-url":"http://www.google.md/#q=Nicolas+Steno&ct=steno12-hp&oi=ddle&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=8c5a975d815425a&biw=1920&bih=881"
                },
                "visibility":{"code":"anyone"}
        }
           
 
resp, content = client.request(url, 'POST', body=simplejson.dumps(body), headers={'Content-Type':'application/json'})
print resp
print content

Robert Schuon

Joined: 07/19/2011

#5 Mon, 02/06/2012 - 17:04

[Mon Feb 06 19:57:06 2012] [error] {'status': '400', 'content-length': '295', 'cneonction': 'close', 'vary': '*', 'server': 'Apache-Coyote/1.1', 'date': 'Tue, 07 Feb 2012 00:57:05 GMT', 'x-li-request-id': 'US6U68SRDJ', 'x-li-format': 'xml', 'content-type': 'text/xml;charset=UTF-8'}
[Mon Feb 06 19:57:06 2012] [error] <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
[Mon Feb 06 19:57:06 2012] [error] <error>
[Mon Feb 06 19:57:06 2012] [error] <status>400</status>
[Mon Feb 06 19:57:06 2012] [error] <timestamp>1328576226113</timestamp>
[Mon Feb 06 19:57:06 2012] [error] <request-id>US6U68SRDJ</request-id>
[Mon Feb 06 19:57:06 2012] [error] <error-code>0</error-code>
[Mon Feb 06 19:57:06 2012] [error] <message>Couldn't parse share document: error: Unexpected end of file after null</message>
[Mon Feb 06 19:57:06 2012] [error] </error>
[Mon Feb 06 19:57:06 2012] [error]

That is still what I get. I'll try your sample body, but after that, I don't know what else to try...

Robert Schuon

Joined: 07/19/2011

#6 Mon, 02/06/2012 - 17:18

Here's the body, response and content using your sample body:

[Mon Feb 06 20:08:56 2012] [error] {'comment': 'Posting from the API using JSON', 'content': {'submitted-url': 'http://www.google.md/#q=Nicolas+Steno&ct=steno12-hp&oi=ddle&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=8c5a975d815425a&biw=1920&bih=881'}, 'visibility': {'code': 'anyone'}}
[Mon Feb 06 20:08:56 2012] [error] {'status': '401', 'content-length': '393', 'vary': '*', 'server': 'Apache-Coyote/1.1', 'date': 'Tue, 07 Feb 2012 01:08:56 GMT', 'x-li-request-id': '455N0ZMIPS', 'x-li-format': 'xml', 'content-type': 'text/xml;charset=UTF-8'}
[Mon Feb 06 20:08:56 2012] [error] <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
[Mon Feb 06 20:08:56 2012] [error] <error>
[Mon Feb 06 20:08:56 2012] [error] <status>401</status>
[Mon Feb 06 20:08:56 2012] [error] <timestamp>1328576936475</timestamp>
[Mon Feb 06 20:08:56 2012] [error] <request-id>455N0ZMIPS</request-id>
[Mon Feb 06 20:08:56 2012] [error] <error-code>0</error-code>
[Mon Feb 06 20:08:56 2012] [error] <message>[unauthorized]. OAU:aeVqbOl_04fi-NttaxUf674unzS_U5BQEA_BfDHfwByfkd3XG-pHVoqUbkr-teST|5ce4470f-d124-41f0-b5a9-dc423abbe070|*01|*01:1328576936:5ip9Wft3w1J5q6I+SpFNc/hpNLs=</message>
[Mon Feb 06 20:08:56 2012] [error] </error>

As far as I can tell now, I replicate your code snippet exactly, with our info. The authorization of users works perfectly, I just don't know why the sharing fails.

import oauth2 as oauth
try:
import json
except ImportError:
import simplejson as json

def post_status(self):
body = {"comment":"Posting from the API using JSON",
"content":{
"submitted-url":"http://www.google.md/#q=Nicolas+Steno&ct=steno12-hp&oi=ddle&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=8c5a975d815425a&biw=1920&bih=881"
},
"visibility":{"code":"anyone"}
}
share_url = 'http://api.linkedin.com/v1/people/~/shares'
consumer = oauth.Consumer(settings.LINKED_IN_API_KEY, settings.LINKED_IN_API_SECRET)
client_token = oauth.Token(self.oauth_token, self.oauth_token_secret)
client = oauth.Client(consumer, client_token)
json_content = json.dumps(body)
headers = {"Content-Type":"application/json"}
response, content = client.request(share_url,'POST',json_content, headers=headers)
print body
print response
print content
return response

That's the whole thing. Any ideas where I messed up?

Thanks,
Bob

Robert Schuon

Joined: 07/19/2011

#7 Tue, 02/07/2012 - 05:53

I tried to edit my post, but it returns access denied, so I fixed one problem, but it still refuses to post:

import oauth2 as oauth
try:
import json
except ImportError:
import simplejson as json

def post_status(self):
body = {"comment":"Posting from the API using JSON",
"content":{
"submitted-url":"http://www.google.md/?q=Nicolas+Steno&ct=steno12-hp&oi=ddle&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=8c5a975d815425a&biw=1920&bih=881"
},
"visibility":{"code":"anyone"}
}
share_url = 'http://api.linkedin.com/v1/people/~/shares'
consumer = oauth.Consumer(settings.LINKED_IN_API_KEY, settings.LINKED_IN_API_SECRET)
client_token = oauth.Token(self.oauth_token, self.oauth_token_secret)
client = oauth.Client(consumer, client_token)
response, content = client.request(share_url,'POST',body= json.dumps(body), headers={"Content-Type":"application/json"})
print body
print response
print content
return response

Got a 401 from that post.

Thanks,
Bob

Kirsten Jones

Joined: 06/30/2011

#8 Tue, 02/07/2012 - 11:11

I'm not sure why your request is failing. The best way to debug issues is using the strategies on Debugging API Calls. I'm not sure how, but your headers or what's being sent must be different.

Here's the information from my post:
Request headers:

user-agent: Python-httplib2/$Rev$
Host: api.linkedin.com
authorization: OAuth realm="http://api.linkedin.com", oauth_body_hash="JtgCKBurLIPLM4dXkn2E3lgrfI4%3D", oauth_nonce="33660894", oauth_timestamp="1328641690", oauth_consumer_key="xxx", oauth_signature_method="HMAC-SHA1", oauth_version="1.0", oauth_token="xxxx", oauth_signature="yk9oNHEL3FksrzH1MB7Mo4ceHfE%3D"
content-type: application/json
Content-Length: 239
Accept-Encoding: identity

POST Data:

{"comment": "Posting from the API using JSON", "content": {"submitted-url": "http://www.google.md/#q=Nicolas+Steno&ct=steno12-hp&oi=ddle&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=8c5a975d815425a&biw=1920&bih=881"}, "visibility": {"code": "anyone"}}

You can use HTTPScoop or Fiddler to do this debugging - hopefully it'll help with your issue. Another possibility is that you're using the request_token token information and not exchanging that request token for a long-lived access token, in which case you can't make regular requests either.

Robert Schuon

Joined: 07/19/2011

#9 Tue, 02/07/2012 - 11:37

The header is being changed (I assume by OAuth2) to the wrong header.

[Tue Feb 07 14:22:10 2012] [error] [********************LinkedIn API Diagnostics**************************]
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error] |-> Status: 400 <-|
[Tue Feb 07 14:22:10 2012] [error] |-> "\\n** Status:\\n\\tA 400 response was received. Usually this means your request was formatted incorrectly or you added an unexpected parameter." <-|
[Tue Feb 07 14:22:10 2012] [error] |-> Key: My Key Removed <-|
[Tue Feb 07 14:22:10 2012] [error] |-> URL: http://api.linkedin.com/v1/people/~/shares <-|
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error] [*****Sent*****]
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error] |-> Headers:{"Content-Type": "application/x-www-form-urlencoded"} <-|
[Tue Feb 07 14:22:10 2012] [error] |-> Body: {"comment": "Testing out the LinkedIn REST Share API with JSON", "content": {"submitted_url": "http://sanfrancisco.bizjournals.com/sanfrancisco/stories/2010/06/28/daily34.html", "submitted_image_url": "http://images.bizjournals.com/travel/cityscapes/thumbs/sm_sanfrancisco.jpg", "title": "Survey: Social networks top hiring tool - San Francisco Business Times"}, "visibility": {"code": "anyone"}} <-|
[Tue Feb 07 14:22:10 2012] [error] |-> Method: POST <-|
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error] [*****Received*****]
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error] |-> Response object: {"status": "400", "content-length": "295", "cneonction": "close", "vary": "*", "server": "Apache-Coyote/1.1", "date": "Tue, 07 Feb 2012 19:22:09 GMT", "x-li-request-id": "D8A9P3RF48", "x-li-format": "xml", "content-type": "text/xml;charset=UTF-8"} <-|
[Tue Feb 07 14:22:10 2012] [error] |-> Content: <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
[Tue Feb 07 14:22:10 2012] [error] <error>
[Tue Feb 07 14:22:10 2012] [error] <status>400</status>
[Tue Feb 07 14:22:10 2012] [error] <timestamp>1328642530477</timestamp>
[Tue Feb 07 14:22:10 2012] [error] <request-id>D8A9P3RF48</request-id>
[Tue Feb 07 14:22:10 2012] [error] <error-code>0</error-code>
[Tue Feb 07 14:22:10 2012] [error] <message>Couldn't parse share document: error: Unexpected end of file after null</message>
[Tue Feb 07 14:22:10 2012] [error] </error>
[Tue Feb 07 14:22:10 2012] [error] <-|
[Tue Feb 07 14:22:10 2012] [error]
[Tue Feb 07 14:22:10 2012] [error] [******************End LinkedIn API Diagnostics************************]

Do you have any ideas why this might be? I add the headers exactly as given in the tutorial.py for json requests.

Thanks,

Bob

Jeremy Johnstone

Developer Advocate

Joined: 04/17/2011

#10 Tue, 02/07/2012 - 14:21

Hi Robert,

Would it be possible for you to paste the entirety of your code you are working on (the LinkedIn specific bits anyway).

http://gist.github.com is a great way to do that.

-Jeremy

Robert Schuon

Joined: 07/19/2011

#11 Tue, 02/07/2012 - 16:12

Jeremy,

I definitely can, but I'm not sure it will help. We run an out-of-the box OAuth setup that works on Facebook and Twitter, but I went in and added print statements to our log, and the signatures do not match. How is that possible? I had to override some code in oauth2.utils to allow a Content-Type other than "application/x-www-form-urlencoded", and everything else seems perfect, but the signature from your test suite and oauth do not match, no matter what header is used.

Anyway, here's the link: https://gist.github.com/1763267

Thanks,

Bob

Kirsten Jones

Joined: 06/30/2011

#12 Wed, 02/08/2012 - 09:36

Can you try downloading the version of oauth2 we have linked on the Quick Start Guide, and using the code there with your keys, and see if that works for you? If so, you should be able to determine where the programmatic difference is.

Robert Schuon

Joined: 07/19/2011

#13 Wed, 02/08/2012 - 14:53

Yup. I'll try that as soon as I get a second. I'm just confused, because we can log in to Linked In with OAuth2, so why not post?

Bob

Steven Citron-Pousty

Joined: 08/02/2011

#14 Thu, 02/09/2012 - 18:27

Robert:
Can you do a simple GET against your profile?
http://api.linkedin.com/v1/people/~

Thanks
Steve

Robert Schuon

Joined: 07/19/2011

#15 Mon, 02/13/2012 - 05:25

@Steve,

Yes, we can, because that is how we get the users profile info for our database. I had created/deleted my info several times during testing, and every time it works fine. However, when I try the POST request, it fails. I will be trying the recommended OAuth today, if I can get it to install properly. Historically, OAuth code is very hard to install in Python, and I run Python 2.6/Django 1.3 under Windows 7 on my dev laptop, which is extra hard. Our server is a standard Linux/Apache box, so it's actually easier in some ways. I will keep you up to date on my progress.

Thanks,

Bob

Robert Schuon

Joined: 07/19/2011

#16 Mon, 02/13/2012 - 10:55

So.....
I upgraded to OAuth 1.5.221, and forced the header to match, and the signatures match, and it posts correctly. You need to warn people that they must upgrade to 1.5.221, because the earlier versions urlize the signature string, and your server rejects that. I was getting a 401, even though the signatures matched, because it was getting urlized, i.e. the / became %2F, and the = became %3D.

Now I can post, and I am very happy! thank you for all of your help!

Bob

Jeremy Johnstone

Developer Advocate

Joined: 04/17/2011

#17 Tue, 02/14/2012 - 13:04

Hi Bob,

Thanks for the tip!

-Jeremy

Ibraheem Khan

Joined: 10/25/2011

#18 Wed, 02/22/2012 - 13:04

Excellent stuff! I've been following this thread since I started it and I think this is the only place on the internet where this particular issue has been discussed. I apologise for taking so long to reply; between my final year project and work as a web dev, I'm tied up half the time!

But seriously, thank you guys so much! I will test out what's been posted, see if I can change to oAuth2 as it seems more straight-forward for what I'm trying to do.

You lot are legends. :)

LinkedIn Developers

Search form

Python REST API oAuth Problem