twitter

Spam–the junk message type, not the tasty luncheon meat from Hormel (pan fried and served on rice, it really can’t be beat…)–is a significant annoyance on all the “Web 2.0″ services, but nowhere more so than on Twitter. Twitter meets all the criteria for an easy spam target:

• Low barriers to entry: scores of free accounts can be easily and automatically created through the Twitter API
• Large audience: some 19.2 million users in October 2009, according to ComScore
• A pervasive culture of implied trust and promiscuous sharing

If spam were a viral infection vector, Twitter would be an over-crowded, fetid swamp where not nearly enough people wash their hands.

The Stop Twitter Spam site maintains a list of tools for combating spam, and also a list of feature proposals (many of which have been implemented) that could help mitigate the problem. And while these are all good tools and useful suggestions, they’re not a panacea: the Twitter spam werewolf is impervious to silver bullets, because the open nature of the service makes spamming so cheap and easy.

Short of increasing the barriers to entry, limiting the ease of communication, or restricting the number of messages that can be sent–all things that would cripple Twitter in its current incarnation–there’s not much that can be done in a centralized fashion to solve the problem. Instead, the cost of fighting the spam lands largely on the non-spamming users. And there’s where the battle will ultimately be won or lost.

Our best weapon against Twitter spam (and other Internet spam sources) is healthy suspicion, just short of paranoia. It’s far too easy to be lulled by the fluid back-and-forth of Twitter communication into forgetting to ask, “Did this person wash their hands? What nasty germs might be lurking behind that URL?” We need to be vigilant about Twitter spam, which can come in direct messages, @replies, or follows, in order to block its spread; since much of the spam comes from accounts hijacked through “phishing” scams, making sure our own accounts are clean is a good defense against infection.

My own guidelines in assessing followers, links, and other Twitter content include:

• The follower/following ratio and number of tweets: now a part of the “new follower” notification, this is the easiest way to catch the spambots. Though Twitter tends to be asymmetrical, with most users following more people than follow them, a ratio that is too heavily weighted toward following–say, following 20 people for every one that follows them–is a clear indication that something is amiss. If that account is also not doing much tweeting–one or two messages, especially about making your teeth whiter or earning $325.42 a day or finding love in all the wrong places–then you’ve almost certainly been targeted by a spambot. Use the “Report as Spam” button to block the account and move the issue up the chain, and then get on with your life.
• The profile picture: If the account is using the default bird icon, they may be a spambot, or they may just be someone who hasn’t updated their profile yet. If the account is using a picture of a hot babe, shown from chin to bellybutton, then the odds are approaching certainty that this account is up to no good. If the account has the hot-babe profile and tweets links to weight loss miracles, the odds need not be calculated; “Report as Spam” and proceed.
• Grammar and content: Spambots don’t do so well on standardized tests of written communication. If the tweet has a lot of misspellings and webby abbreviations, the related link is suspect. If the message is “LOL u gots to c thisss” then it’s certainly spam. Don’t click.
• Context and consistency: Know the people you follow, and who follow you. If you’re following someone because of their pithy observations about Romanian poetry, or their scathing reviews of trends in script-based computer language design, and they suddenly send you a link to a Cialis offer, something is likely amiss. Out of courtesy, let them know, so they can take steps to resolve the problem; if they continue to send spam, then they’re either not taking the problem seriously, not being careful about their account, or have gone over to the Dark Side; unfollow at least, block if you must, report if you suspect a permanent hijacking.
• Look before you link: The 140-character limit of tweets is a great enabler of scary links. With the average URL length coming in at 34 characters (at least according to Kelvin Tan’s calculations), and with many news and blog sites’ URLs even longer, there’s been a proliferation of URL shortening services. Alas, it’s far too easy for malicious content to hide behind a generic shortened URL. (And this assumes that the shortener itself isn’t compromised in some way; the security implications of third-party URL shortening are really quite scary…) LongURL (available as a web site service, Firefox plugin, Greasemonkey script, or jQuery plugin), expandurl (web site and API), and ExpandMyURL (web site and bookmarklet) help by letting you preview the original URL before loading its content into your browser. If you’re not using one of these services, or a similar tool for previewing shortened URLs, you’re putting yourself and others at risk.
• Don’t be part of the problem: Keep your own account from passing spam around, or giving the impression of spamminess. Don’t give out your Twitter credentials to any service that you don’t completely trust; shorten URLs only when truly necessary; provide context for your links (“Great discussion of Hegelian epistemology” rather than “U shd c this,” though you’ll probably need to use that URL shortener after all); be as trustworthy and consistent in your tweeting as you strive to be in the physical world.
• Follow smart people: I don’t get a lot of spammy or suspicious junk on Twitter, largely because I’m careful to filter for quality up front. It’s not hard to spot the garbage when so much of the content is high quality, and smart people are less likely (I hope) to be drawn into a phishing scam, or will at least do something about it if their accounts are compromised. Be ruthless about unfollowing accounts that are frequently hijacked; it may seem cruel, but quarantine can be a useful tool in fighting virtual diseases too.

It’s a shame that the cost of combating Twitter spam falls back on the users, but that’s the paradox of the free and open model that makes Twitter so valuable to begin with. As in a free society, where we occasionally trade comfort and security for liberty, keeping Twitter from being overrun with spam is one of the responsibilities of its users.

As I’ve mentioned before, I’ve been using Furl as part of my Twitter “strategy.” I go through my RSS feeds a couple times a day, flagging the items that might be of interest to my Twitter followers with Furl’s easy-to-use bookmarklet and categorizing them according to the appropriate Twitter identity: Minnesota things for We Like It Here, poetry and Dickinson-related items for Daily Dickinson, and various odds and sods for my personal Twitter account. Then my good friends at twitterfeed take over, periodically sending the resulting RSS feeds to the appropriate Twitter zone.

Alas, no longer.

I went to add this lovely spring photograph to the We Like It Here feed, and the bookmarklet responded with an abrupt message about how Furl has been “absorbed” by something called Diigo. There was no warning that I saw that this was coming (the Furl blog has been silent since January 21, and the Twitter feed since January 6); it just happened. At least the notice was nice enough to include instructions on how to transfer my feed contents over to the new service, and the transfer was flawless, but it was an abrupt interruption to my workflow.

One of the things I liked about Furl was that it was bare bones and no-nonsense. If you want to have a categorized collection of bookmarks, stored on the web for easy access from any location and available as RSS feeds, with the ability to do some simple annotation, it was perfect. No voting, no “Digging,” no “friending,” just bookmarking.

Diigo seems to want to be more than that. Their front page announces that “reading is more fun with friends” (an assertion with which I beg to differ, being a proudly solitary reader), that “[y]ou are what you annotate” (actually, I’m not…), and gives me all sorts of opportunities to “meet people” and “join communities.” Honestly, I’ve met about as many people as I care to, and I don’t need another “social media” site to worry about; Flickr, Twitter and Facebook are sufficient for my curmudgeonly needs.

Still, Diigo did a good job of importing my bookmarks, and they have a nice enough Firefox plugin for doing the bookmarking, and they publish my collections as RSS. (I had to make each one public and set the sorting to be descending by date, but that was reasonable enough.) It’s an interruption, but not a catastrophe.

But there was still one more fly in the ointment: Diigo adds stuff the RSS items that I don’t especially want. I discovered that, in addition to the title and URL (which is all I really want), the feed coming over to Twitter included “Tags: no_tag Posted by: mhartford” on each item. With 140 characters allowed for each “tweet,” that’s an expensive chunk of useless information. (Incidentally: if my bookmark doesn’t contain any tags, wouldn’t it make sense that the who “Tags:” element be excluded from the RSS item? If I were writing the code, that’s what I’d do…)

Fortunately, there’s a good (and fun) solution: Yahoo! Pipes.

Pipes is an online feed manipulation tool that can consume RSS, HTML snippets, web service output, and other kinds of content, perform operations on the incoming content, and emit the output as a new RSS feed (or as JSON, e-mail messages, or a chunk of PHP code, if one is so inclined). The resulting collection of actions and its output can then be saved and published.

I looked at the XML that Diigo spat out, and discovered that it had some blocks of HTML inside the description element that I wanted to get rid of:

<description>
<p></p>
<p><strong>Tags:</strong> <a href='http://www.diigo.com/user/mhartford/no_tag' rel='tag'>no_tag</a>
</p><p><strong>Posted by:</strong> <a href='http://www.diigo.com/user/mhartford'>mhartford</a></p>
</description>


By creating a Yahoo! Pipe pointing to the Diigo RSS URL, and passing it through a regex (“regular expression”) control, I was able to remove the offending crud. The regex control simply points at the description item, and asks Yahoo! Pipes to remove anything that matches <p.*></p>. Hey presto! And it’s clean.

A useful side effect of the Yahoo! Pipes approach is that I’ve essentially created a proxy service for my RSS feeds. If I should have to migrate again, or otherwise fiddle with the output, I don’t have to point my feed readers or twitterfeed to another URL: I can continue to point to Yahoo! Pipes and make drastic changes invisible to consumers.

I’m still waiting to be impressed by Diigo; I don’t have any particular interest in the social media aspects of it, and I worry that if that component doesn’t grab traction in an increasingly over-”friended” online world, the useful aspects will be lost when Diigo goes under. I’ll continue to look for a Furl replacement.

But Yahoo! Pipes impresses me to no end. It’s easy and fun to work with (you drag controls around and string them together in a nice UI), powerful as heck but simple to use. I expect it to be a key piece of my arsenal of tools for taming the Internet’s ever-expanding river of content.

I’ve recently become interested in Twitter, the “micro-blogging” service. The 140-character limit, the variety of items published, and the transience of its topics make it very different from most “social media” sites, RSS feeds, or “traditional” blogs.

Particularly intriguing about Twitter, from a technology point of view, is the rich ecosystem of services built around it. There are hundreds, maybe thousands, of web applications that feed or read twitter; a quick perusal of the “tweets” I’m following show that a minority are coming from the Twitter website itself; many more are published through third-party tools.

Here are a few of the Twitter tools that I’ve found especially helpful for the way I use Twitter.

Splitweet

I maintain three different Twitter identities: my personal account, DailyDickinson for my Emily Dickinson site, and WeLikeItHere for my Minneapolis-St. Paul blog, We Like It Here. In addition to publishing links to blog posts and interesting sites, I use these accounts to learn about the topics that interest me (literature, photography, programming, Minnesota) and to maintain contact with like-minded people and institutions.

The Twitter site itself isn’t very easy to use with multiple personalities; there’s a single session at a time, so you need to log off and log back in to “become” a different Twitter user. It’s also hard to follow different sets of Twitter feeds on Twitter itself; you’re bound to the feeds of your current identity, and need to switch identities to be sure not to miss something.

That’s where Splitweet is especially useful. Splitweet lets you create a “master” account to which you bind multiple Twitter identities, and see all of your incoming tweets in a single feed. You can turn individual feeds off and on, so temporarily becoming one of your identities is very easy. The interface lets you know which identity each tweet belongs to, and also lets you post tweets as one or more of your identities; re-tweet, reply, or DM on each tweet in your combined feed; and see all incoming replies and DMs in a combined list.

There are a few things that I wish Splitweet offers that it doesn’t. Follower/following management, for example, isn’t part of the interface; I still have to log in to Twitter with my separate identities to manage my connections. And it would be great if it included some of the features of the other tools I use so I’d have just one location to visit. But all in all, it’s an indispensable site for people with multiple personalities.

HootSuite

HootSuite (née BrightKit) is also geared to the multiple-identity Twitter user. You can register all of your Twitter identities, and schedule “tweets” from each. It also allows multiple “editors” for each Twitter account; this would be a useful feature for a group blog, where several people are granted permission to “tweet” from the same Twitter account.

In addition to manually scheduling your “tweets,” HootSuite offers an RSS-feed integrator that will publish feed entries to Twitter. I tried this early on, and found that the service was flaky; apparently their infrastructure was overwhelmed by the service’s popularity, but recent upgrades may alleviate this problem.

twitterfeed.com

Meanwhile, I’ve been using twitterfeed.com for RSS integration. After logging in (through the OpenID interface), you can link multiple Twitter accounts to RSS feeds, and schedule “tweets” from those feeds.

I’ve been using Furl as my “social bookmarking” site for a couple years, because it allows you to build RSS feeds of your bookmarks. By putting links of interest to the followers of each of my IDs into the appropriate Furl category, I can keep my Twitter feeds up to date with relevant information while keeping up with the news. (I use the Furl plug-in for Firefox to make it that much simpler; right-click the page, decorate the link title with appropriate Twitter “#” and “@” info, add to a category, and hey presto! it’s done.)

Twitter Tools for WordPress

Alex King has provided a wealth of plug-ins to the WordPress community; one of the most recent, Twitter Tools, provides easy integration with Twitter.

I use this plug-in to send a “tweet” with each new blog post, and to add a list of recent “tweets” to a couple side bars. It also provides a daily or weekly digest of “tweets” as blog posts.

The automatic “tweet” only happens, though, from the WordPress UI; if you schedule a post for future publication, no tweet is fired, even if the “notify twitter” option is selected. This is where the HootSuite scheduling is handy; I typically put together the Daily Dickinson posts some days in advance, so I can use HootSuite to send those tweets to coincide with the blog publication. It’s a little extra effort, but worth it to keep the Daily Dickinson audience up to date. (I suppose I could also use the RSS feed integration from HootSuite or twitterfeed.com, but I like the targeted timeliness of the scheduled tweet).