Archive

Posts Tagged ‘Flickr API’

Plotting points on an OpenStreetMap Export

February 24, 2010 Leave a comment

This map shows the location of pictures in a Flickr group superimposed on an OpenStreetMap (OSM) export.

If you try plotting points directly on an OSM map, you’ll find that points are all over the shop. The reason is that OSM exports use the Mercator projection; you need to change the latitude and longitude coordinates into the Mercator projection.

Basemap to the rescue!

If you use the code in the previous post, you can change the plotting code.

from basemap import Basemap 
import matplotlib.pyplot as plt
import matplotlib.pylab as pylab
import numpy as np
import string
import matplotlib.cm as cm

x=[] #longitudes
y=[] #latitudes

fi=open(r'C:\infoviz\scotland_photos.csv','r')

linenum=0
for line in fi:
    if linenum>0:
        line=string.replace(line, "\n","")
        try:
            fields=string.split(line,",")
            lon,lat=fields[0:2]
            x.append(float(lon))
            y.append(float(lat))
        except:
            print "Error!"
    linenum+=1
fi.close()

m = Basemap(llcrnrlon=-8.0,llcrnrlat=54.5,urcrnrlon=1.5,urcrnrlat=59.5,lat_ts=20,
            resolution='h',projection='merc',lon_0=-4.36,lat_0=54.5)
x1,y1=m(x,y)
m.drawmapboundary(fill_color='white') # fill to edge
m.scatter(x1,y1,s=5,c='r',marker="o",cmap=cm.jet,alpha=1.0)

Rather than using basemap to draw the outline of Scotland, this script simply creates a scatter plot on a white background, like so:-

Now, you need to export the map from OpenStreetMap.

The corners have been set as follows..

… llcrnrlon=-8.0,llcrnrlat=54.5,urcrnrlon=1.5,urcrnrlat=59.5 …

llcrn stands for the lower-left coordinate, and and urcrn for the upper-right coordinate.

So if you if you export from OpenStreetMap using these coordinates…

.. you’ll have a map of Scotland with the Mercator projection.

The two images can then be composited in Photoshop or another graphics app like the Gimp.

Advertisements

Creating a pinboard map of geotagged photos in a flickr pool

January 14, 2010 1 comment

In this post I’ll show how to produce a simple pinboard map of geotagged photos in a flickr group pool, using Python and Basemap/Matplotlib. You’ll need:-

There are two short scripts here:-

  • A script to find the longitude and latitude of geotagged photos in the group pool
  • A script to generate the plot

The first script produces a CSV file; the second uses this CSV file to produce the plot.

Here’s the script to produce the CSV file with photo locations:-

# -*- coding: UTF8 -*-
'''
Created on 12 May 2009
Based on beejs flickr API
Produce a list of photo locations for a given
group's pool on flickr
@author: Steven Kay
'''

import flickrapi
import string
import datetime
import string
import time

# Enter your API key below
# You can apply for an API key at 
# http://www.flickr.com/services/apps/create/apply
api_key = '' 

# paste group NSID below
group = '1124494@N22'

# sample... fetch your latest images with 
# a count of the views, faves and comments

if __name__ == '__main__':
    flickr = flickrapi.FlickrAPI(api_key)
    response_photos = flickr.groups_pools_getPhotos(group_id=group,per_page=500,extras='geo')
    root=response_photos.findall('.//photos')
    pages=int(root[0].get('pages'))
    if pages>8:
        # stop after 8 pages of 500 images
        # not sure if groups.pools.getPhotos has the same
        # 4000 image limit as photos.search..?
        pages=8
    
    fo=open(r"C:\infoviz\scotland_photos.csv","w")
    print "Longitude,Latitude"
    fo.write("Longitude,Latitude\n")
    for page in range(0,pages):
        response_photos = flickr.groups_pools_getPhotos(group_id=group,per_page=500,page=str(page),extras='geo') 
        for photo in response_photos.findall(".//photos/photo"):
            try:
                lat=photo.get('latitude')
                lon=photo.get('longitude')
                st="%s,%s" %(lon,lat)
                if not st=="0,0":
                    # ignore the odd buggy 0,0 coords
                    print "%s,%s" %(lon,lat)
                    fo.write("%s,%s\n" %(lon,lat))
            except:
                pass
        time.sleep(1)
    fo.close()

You’ll need to find the NSID of the group as an input; you can find this with the flickr API call flickr.group.search.

Now, you have a simple CSV file with the latitude and longitude of each geotagged image in the pool.

Longitude,Latitude
-5.167792,58.352519
-4.024359,57.675544
-4.230251,57.497356
-4.2348,57.501045
-4.84703,56.646034
-4.306168,55.873986
-3.586263,56.564732
...

This demo uses the Photography Guide to Scotland pool.

The next step is to plot the map.

'''
Simple Matplotlib/Basemap pinboard map for
Flickr Groups.

Need to provide a CSV file in following format

Longitude,Latitude
20.1,-3.25
20.225,-3.125
.. etc..

Created on 10 Oct 2009

@author: Steven Kay
'''

from basemap import Basemap 
import matplotlib.pyplot as plt
import matplotlib.pylab as pylab
import numpy as np
import string
import matplotlib.cm as cm

x=[] #longitudes
y=[] #latitudes

fi=open(r'C:\infoviz\scotland_photos.csv','r')

linenum=0
for line in fi:
    if linenum>0:
        line=string.replace(line, "\n","")
        try:
            fields=string.split(line,",")
            lon,lat=fields[0:2]
            x.append(float(lon))
            y.append(float(lat))
        except:
            pass
    linenum+=1
fi.close()

# cass projection centred on scotland
# will need to replace with a projection more suited
# to the group you're plotting

m = Basemap(llcrnrlon=-8.0,llcrnrlat=54.5,urcrnrlon=1.5,urcrnrlat=59.5,
            resolution='h',projection='cass',lon_0=-4.36,lat_0=54.5)
x1,y1=m(x,y)
m.drawmapboundary(fill_color='cyan') # fill to edge
m.drawcountries()
m.drawrivers() # you may want to turn this off for larger areas like continents
m.fillcontinents(color='white',lake_color='cyan',zorder=0)
m.scatter(x1,y1,s=5,c='r',marker="o",cmap=cm.jet,alpha=1.0)

plt.title("Photography Guide to Scotland in FlickR") # might want to change this!
plt.show()

This script uses a projection centred around scotland; you’ll need to change the following line…

m = Basemap(llcrnrlon=-8.0,llcrnrlat=54.5,urcrnrlon=1.5,urcrnrlat=59.5,
            resolution='h',projection='cass',lon_0=-4.36,lat_0=54.5)

…to something more suitable for your needs. Basemap provides an intimidating list of projections which should meet your needs.

Using Flickr API to get the views, faves and comments of your most popular images

September 25, 2009 Leave a comment

One of the first things I wanted to do with the Flickr API was to get some stats on my most popular images.

You can get this info through the web front end, but there’s no option to download the stats in delimited format (such as CSV) so it can be analysed in a spreadsheet.

I wanted to work out if there was a pattern emerging in the key stats for my 200 most popular images…

  1. Number of Views
  2. Number of Favourites
  3. Number of Groups posted to
  4. Number of sets an image is in

Using a Python script (v2.5) and Beej’s FlickR API, this is fairly straightforward. It doesn’t require authentication.

The script runs slowly as it ‘plays nice’, leaving a seconds pause between calls, courtesy of the time.sleep() function. I don’t want to thrash the server.

# -*- coding: UTF8 -*-

import flickrapi
import datetime
import time
import string

# enter your api key below
api_key = 'PUT_YOUR_API_KEY_HERE' 

# enter the user id below (you can use flickr.people.findByUsername to get this for any user)
# it'll look something like 99999999@N99
userid='USER_ID_TO_SEARCH'

# delimiter. Use comma if you want, I tend to use ~
DELIMITER="~"

# dump number of views in delimited format

if __name__ == '__main__':
    #output format : "photoid,title,views,faves,groups,sets"
    flickr = flickrapi.FlickrAPI(api_key)
    photos = flickr.photos_Search(user_id= userid,extras='views', per_page='200', page='1', sort='interestingness-desc')
    for photo in photos.find('photos'):

        title = string.replace(photo.get('title'),",","") #in case you want to use comment as a delimiter ;0)
        
        # number of views
        id = photo.get('id')
        views=photo.get('views')
        
        # fave count (up to 50)
        faves = flickr.photos_getFavorites(photo_id=id,per_page=50)
        countfaves=faves.find('photo').get('total')
        time.sleep(1)
        
        # pools and sets posted to
        contexts=flickr.photos_getAllContexts(photo_id=id)
        posted_groups=len(contexts.findall('.//pool'))
        posted_sets=len(contexts.findall('.//set'))
        time.sleep(1)
        
        # comments
        comments=flickr.photos_comments_getList(photo_id=id)
        countcomments=len(comments.findall('.//comment'))
        
        # output as delimited text
        tokens=(id,title,str(views),str(countcomments),str(countfaves),str(posted_groups),str(posted_sets))
        converted=DELIMITER.join(tokens)
        
        print converted

This script dumps to the console, rather than a file; but it’s easily modified to write to a file. It should work with comma as a separator (for CSV use) as the title tag is stripped of commas…

Here’s some sample output from my photostream…

The format is :
photo id~title~views~favourites~groups~sets


2113237108~north-berwick-old-pier~259~25~21~6~3
3598511429~paris photo heatmap~736~6~12~5~3
3609118442~heart texture~861~10~26~0~2
2304836447~persistence de-motivator~4964~1~4~1~1
2347673075~bergen-ole-bull-plass-lensbaby~184~12~7~6~4
1621047086~banners-down-princes-street~325~11~8~4~2
3688253826~St Anthony's Chapel Edinburgh~124~15~15~9~1
2717978614~st marys~88~9~7~3~2

Once you have the output saved to a text file, you can import it into a spreadsheet (like OpenOffice or Excel) and play around with the figures 🙂

mapping flickr group activity

September 20, 2009 Leave a comment

flickr group activity visualization

Mapping the activity levels of approx 1,500 Flickr groups against the number of members of each group, using the Flickr API.

The x axis is the group size, the y axis is the number of seconds an image can expect to stay on the ‘front page’. This was measured as the timestamp difference between the 1st and 13th images in the pool (the landing page for a group shows 12 images; the user has to follow a link to show more).

Note that the image is a log-log plot, as the groups follow a power-law distribution.

Groups in the bottom right are the busiest – this includes the B&W and FlickrCentral groups. Groups in the top right have lots of members but are more vigorously moderated, so images are added to the pool more slowly (or new submissions are deleted). The group touching the bottom of the graph is the “10 million images” group, where users are encouraged to “dump and run”.

This is a hexbin plot – the colour represents the number of groups falling within a certain range of values. Red=Lots, Green=Fewer, Blue=Few.

As you’d expect, larger groups tend to have a higher turnaround of images, but there’s a lot of variation.

The most common group size seems to be around 2000-3000 members; a group this size, you can expect an image to stay on the front page for around 2-3 hours. With the largest groups, this drops to around 5 minutes.

Mapping 24 hours of Flickr geotagging in Python

September 6, 2009 Leave a comment

The aim of this project was to find out where in the world people were geotagging their photos on flickR, using the flickR API.

world

The approach taken was to poll one of the flickR ‘pandas’, ‘Wang Wang’. This is a service which keeps track of geotagged photos as they come in.

The following Python script runs in the background, polls the service once a minute, and appends the location of newly tagged photos to a CSV file. It only asks for up to 100 photos in the previous minute; in reality, up to 120 are returned in any one minute! The average was around 80/minute when I last ran this.

The flickr API is being accessed using beej’s flickr api.

# -*- coding: UTF8 -*-
'''
Created on 28 Apr 2009
Ask WangWang for recently geotagged photos
@author: Steven
'''

import flickrapi
import datetime
import string
from time import time as t
import time

api_key = 'YOUR_API_KEY_HERE'
flickr = flickrapi.FlickrAPI(api_key)
if __name__ == '__main__':
    ct=0
    lastct=0
    print "Timestamp  Total   This"
    while True:
        tstamp=int(t())-60
        wangwang = flickr.panda_getPhotos(panda_name='wang wang', interval=60000, per_page=100, last_update=tstamp, extras='geo')
        fo=open("c:\\wangwang24hours.csv","a")
        for x in wangwang.find('photos'):
            s= "%d,%s,%s,%s\n" % (tstamp, x.get('longitude'),x.get('latitude'),x.get('id'))
            ct=ct+1
            fo.write(s)
        time.sleep(60)
        fo.close()
        print "%10s %07d %04d" %(tstamp,ct,ct-lastct)
        lastct=ct
    print 'done'

Once we have the data, it’s time to visualise it. A heatmap seemed a good choice; the chart uses the Matplotlib ‘hexbin’ style. This takes two arrays of the same size (here, the longitudes are in X and the latitudes in Y) and maps the values onto a hexagonal grid (here, of size 180×180), counting the number of photos which fall into each hexagonal bin.

Each bin is coloured according to the number of points that fall into it; red have most, green have less, blue have the least.

The following script takes the output from the previous script, and plots it.

import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
import string

X=[]
Y=[]
fi=open(r"c:\wangwang24hours.csv")
for line in fi:
    ignore,x,y,ignore2=string.split(line, ",")
    if x!='None' and y!='None':
        X.append(float(x))
        Y.append(float(y))
fi.close()
hexbin(X,Y,gridsize=180,bins='log',cmap=cm.jet,linewidths=0,edgecolors=None)
show()