Twitter

You can fetch, store and display data about your own Tweets and Liked Tweets for one or more Twitter accounts. By default the included models and templates will link to image and video files at Twitter, but you can also download the original files to store locally. These can then be used to serve different sizes of images locally.

Set-up

In the Django admin, add a new Account in the Twitter app, with your API credentials from https://apps.twitter.com/ .

Then you must do:

$ ./manage.py fetch_twitter_accounts

which will fetch the data for that Account’s Twitter user.

Tweets

If you have more than 3,200 Tweets, you can only include older Tweets by downloading your archive and importing it. To do so, request your archive at https://twitter.com/settings/account . When you’ve downloaded it, do:

$ ./manage.py import_twitter_tweets --path=/Users/phil/Downloads/12552_dbeb4be9b8ff5f76d7d486c005cc21c9faa61f66

using the correct path to the directory you’ve downloaded and unzipped (in this case, the unzipped directory is 12552_dbeb4be9b8ff5f76d7d486c005cc21c9faa61f66). This will import all of the Tweets found in the archive. The data in the archive isn’t complete, so to fully-populate those Tweets you should run this (replacing philgyford with your Twitter screen name):

$ ./manage.py update_twitter_tweets --account=philgyford

This will fetch data for up to 6,000 Tweets. If you have more than that in your archive, run it every 15 minutes to avoid rate limiting, until you’ve fetched all of the Tweets. This command will fetch data for the least-recently fetched. It’s worth running every so often in the future, to fetch the latest data (such as Retweet and Like counts).

If there are newer Tweets, not in your downloaded archive, then run this:

$ ./manage.py fetch_twitter_tweets --account=philgyford --recent=3200

The 3200 is the number of recent Tweets to fetch, with 3200 being the maximum allowed in one go.

Run this version periodically to fetch the Tweets since you last fetched any:

$ ./manage.py fetch_twitter_tweets --account=philgyford --recent=new

You might also, or instead, want to fetch more than that, eg:

$ ./manage.py fetch_twitter_tweets --account=philgyford --recent=200

This would update data such as the Retweet and Like counts for all of the 200 fetched Tweets, even if they’re older than your last fetch.

If you have more than one Twitter Account in Ditto, the above commands can be run across all of them by omitting the --account option. eg:

$ ./manage.py fetch_twitter_tweets --recent=new

Favorites/Likes

To fetch recent Tweets that that all your Accounts have liked, run one of these:

$ ./manage.py fetch_twitter_favorites --recent=new
$ ./manage.py fetch_twitter_favorites --recent=200

Or restrict to Tweets liked by a single Account:

$ ./manage.py fetch_twitter_favorites --account=philgyford --recent=new
$ ./manage.py fetch_twitter_favorites --account=philgyford --recent=200

Users

When a Tweet of any kind is fetched, its User data is also stored, and the User’s profile photo (avatar) is downloaded and stored in your project’s MEDIA_ROOT directory. You can optionally set the DITTO_TWITTER_DIR_BASE setting to change the location. The default is:

DITTO_TWITTER_DIR_BASE = 'twitter'

If your MEDIA_ROOT was set to /var/www/example.com/media/ then the above setting would save the profile image for the user with a Twitter ID 12345678 to something like this:

/var/www/example.com/media/twitter/avatars/56/78/12345678/my_avatar.jpg

You may periodically want to update the stored data about all the Twitter users stored in Ditto. (quantity of Tweets, descriptions, etc). Do it like this:

$ ./manage.py update_twitter_users --account=philgyford

This requires an account as the data is fetched from that Twitter user’s point of view, when it comes to privacy etc.

Images

Fetching Tweets (whether your own or your Likes) fetches all the data about them, but does not fetch any media files uploaded with them. (It’s not possible to fetch video files (as far as I can tell.) There’s a separate command for fetching images and the MP4 video files for animated GIFs. (There’s no way to fetch the original GIF files.)

You must first download the Tweet data, and then fetch the files for all those Tweets:

$ ./manage.py fetch_twitter_files

This will fetch the files for all Tweets whose files haven’t already been fetched (so, the first time, it’s all the Tweets).

If you want to force a re-fetching of all files, whether they’ve already been downloaded or not:

$ ./manage.py fetch_twitter_files --all

Each image/MP4 is associated with the relevant Tweet(s) and saved within your project’s MEDIA_ROOT directory, as defined in settings.py. There’s one optional setting to customise the directory in which the files are saved. Its default value is as shown here:

DITTO_TWITTER_DIR_BASE = 'twitter'

Files are organised into separate directories according to the final characters of their file names (so as not to have too many in one directory). eg, an image might be saved in:

/var/www/example.com/media/twitter/media/6T/ay/CRXEfBEWUAA6Tay.png

Every uploaded image, animated GIF and video should have a single image. Animated GIFs will also have an MP4 file.

Once you’ve downloaded the original files, you can use these to generate all the different sizes of image required for your site, instead of linking direct to the image files on Twitter. To do this, ensure imagekit is in your INSTALLED_APPS setting:

INSTALLED_APPS = (
    # ...
    'imagekit',
    # ...
)

And add this to your settings.py (its default value is False):

DITTO_TWITTER_USE_LOCAL_MEDIA = True

Any requests in your templates for the URLs of photo files of any size will now use resized versions of your downloaded original files, generated by Imagekit. The first time you load a page (especially if it lists many images) it will be slow, but the images are cached (in a CACHE directory in your media folder).

For example, before changing this setting, the URL of small image (Media.small_url) would be something like this:

https://pbs.twimg.com/media/CjuCDVLXIAALhYz.jpg:small

After choosing to use local photos, it would be something like this:

/media/CACHE/images/twitter/media/Lh/Yz/CjuCDVLXIAALhYz/5a726ea25d3bbd1b35b21b8b61b98c4c.jpg

If you change your mind you can switch back to using the images hosted on Twitter by removing the DITTO_TWITTER_USE_LOCAL_MEDIA setting or changing it to False.

Animated GIFs are converted into MP4 videos when first uploaded to Twitter. Ditto downloads and uses these in a similar way to images. ie, by default the video_url property of a Media object that’s an Animated GIF would be like:

https://pbs.twimg.com/tweet_video/CRXEfBEWUAA6Tay.mp4

If it’s been downloaded and DITTO_TWITTER_USE_LOCAL_MEDIA is True then calling video_url would return a URL like:

/media/twitter/media/6T/ay/CRXEfBEWUAA6Tay.mp4

However, there’s no way to download actual videos that were uploaded to Twitter, and so Ditto will always try to use videos hosted on Twitter, no matter what the value of DITTO_TWITTER_USE_LOCAL_MEDIA.