Actions

Twitter2Mp3: Difference between revisions

From Rabbi Blog

 
(9 intermediate revisions by the same user not shown)
Line 4: Line 4:
Premise: parse Twitter feed using Perl, utilize Text2Wav (Festival) to convert to a wav file and then LAME encode the result to mp3.
Premise: parse Twitter feed using Perl, utilize Text2Wav (Festival) to convert to a wav file and then LAME encode the result to mp3.


=Audio Result=
<flashmp3>http://www.rabbibob.com/cstrike/20090321_twitter.mp3|autostart=no|loop=no|firstTrackName=Sleeper39_1997</flashmp3>
=Command Line=
<pre>
<pre>
text2wave -F 16000 htt.txt -o test2.wav
text2wave -F 16000 htt.txt -o test2.wav
Line 16: Line 12:
</pre>
</pre>


=Text Origin=
=Audio Result=
* Things such as 'h t t p' eventually would be removed.
 
<flashmp3>http://www.rabbibob.com/cstrike/20090324_twitter.mp3|autostart=no|loop=no|firstTrackName=Twitter2Mp3 Example</flashmp3>
 
=Text =
* implemented 'bookmark' - records id from update to file
** read in bookmark and start from that marker (200 max)
* replaced http urls with 'weblink'
* reduced multiple .. to .
* replaced minor emoticons with words
* reversed tweet order (now reads oldest -> newest)
* reports # of tweets since last update (200 max)
* added extra line break - helps text2wav 'breath' between tweets
 
<pre>
<pre>
Oblivion said 15.9 miles down 4.1 to go. 10:59 avg pace. #runners
There have been 200 tweets since the last update
Phreak said @rabbibob you're playing MW - do i have you on facebook?
 
rabbibob said @DAVEfromNY that sucks!
Tweet 1 pralphman said Checking out some film trailers on apple.com Seems there are some fairly good comedies coming in the next few months.
justinrussell said It's funny; whenever I go to choose a font for a project, I always end up with the same 4 or 5. Designers, you may start criticizing me now.
 
chrismarquardt said Voll gut!  http://twitpic.com/2bfq1
Tweet 2 Oblivion said Crap. Yesterday was the first day in 42 that I missed doing at least the body test on the #WiiFit. Foo, its disappointed with me.
DAVEfromNY said WTF?!?! First day of spring it snows, then the second day of spring my furnace dies! Somehow I blame that damn groundhog for all this!
 
dashdrum said Free lunch from @gentryart at 9 Irish Brothers Pub
Tweet 3 dashdrum said @toyotaboy yeah, but my co-workers don't know they aren't clean :-)
rabbibob said @Phreak @tomit I'm slightly addicted to MW atm
 
Phreak said @tomit i think i am, yes. i had about five invites, i accepted them all
Tweet 4 dashdrum said Hot chocolate and pop-tarts.  Breakfast of champions!
tomit said @Phreak I catch the blame for MW's from everyone!  You part of my mafia?
 
cc_chapman said Great video by @cmajor - SXSW in 2 Words -  http://bit.ly/j4pLw
Tweet 5 fortheinsane said @dashdrum what flavour pop tarts?
tomit said Added commenting to what will be my main blog. Much more work to do on it though.  http://bit.ly/cqTCh
 
Oblivion said http://twitpic.com/2beor - 9.75 miles. I lost a glove. Crap.
Tweet 6 draithon said @UberSchwarz u may feel like cattle when they herd u on, but hey for the prices they charge I'm not complaining, ok I want more destinations
Phreak said way to get husband to stay out of the house - tell him he has 50 things to do when he gets home
 
UberSchwarz said Waiting for @draithon to arrive for a day of picture-taking!
Tweet 7 dashdrum said @fortheinsane Brown sugar cinnamon. The best one!
Oblivion said Sweet. Just found a roll of electrical tape on the side of the road. Sunglasses fixed. 8.33 mi 1:34 #runners
 
Phreak said can't think of what it is i'm supposed to be doing other than playing Mafia Wars - damn @tomit
Tweet 8 draithon said I'm overly concerned about a proposal that could cap a persons compensation for their job The interpretations of that law could be disaster
rabbibob said Erin is all signed up for tee-ball this spring!  Callum unfortunately didn't make the minimum age.
 
rabbibob said @UberSchwarz http://tinyurl.com/dgjcnj  TFTTF 175 white balance vid
Tweet 9 draithon said America was built on hard workIf you cap wages, where is motivation going to beHate to say it but greed is a great motivator.
Phreak said glad i'm not a redhead  http://tinyurl.com/d2y2qq
 
UberSchwarz said Not sure I liked the ending to BSG....
Tweet 10 draithon said I'm not in finance, but I can see this proposal being interpreted & construed against other jobs. Ultimately hurting the middle class.
marcapitman said It's sort of depressing to UNLOAD the trash I just loaded into the van. The transfer station is closed on Saturdays until April!
 
Phreak said tweetdeck was acting the maggot and i had to reinstall - got sick of being locked out of my Twitter account for no good reason
Tweet 11 draithon said I usually try to stay away from politics, but the last 18 months - 2 years I have been overly studying & paying better attention to them.
MrTweet said Every user has different purposes for using Twitter, and our aim is to help them do so in the easiest way possible. New posting coming soon!
 
Oblivion said http://twitpic.com/2bc4q - Mile 4.5. Feeling good. Beautiful day, @AmandaPalmer in my ears. #runners
Tweet 12 draithon said lol
Oblivion said Famnm. Just lost the nosepiece to my sunglasses and I forgot spare bandaids.
 
Shultzman said I don't know why @scottsigler is hating on the BSG ending so much.  I liked it.  Watched it twice.  Had something for everyone.
Tweet 13 draithon said Dam @twitterfox is getting aggravatingIt submitted the tweet before I was ready to. Just another thing for this monday I guess
tomit said I love these splash screens. That is just so funny!!! re: http://ff.im/1DnPA
 
merlyn said gonna try dialing back on the oxycontin today to find that sweet spot between wooziness and pain.  already down to 60% of max dose
Tweet 14 adamcurry said Preferred format for promos is mp3
cc_chapman said Dylan at Lacrosse Practice http://twitpic.com/2b9m8
 
Shultzman said RT @jayrosen_nyu All credit to the Washington Post for running Chris Mooney's op ed on George Will's global cooling claims http://is.gd/ojPY
Tweet 15 marcapitman posted Great Foxtrot cartoon on Twitter and parenting Weblink  Props to @trwl
Oblivion said Water planted. Heading out now.
 
cc_chapman said At my first lacrosse practice with Dylan. Can't wait for the games. Fun sport.
Tweet 16 cc_chapman posted SXSW Was a Success For Me - Weblink
UberSchwarz said yeah, you can nest arrays in certain languages.  Its pretty slick
 
Oblivion said Would anyone be interested in tracking my run with Google Latitude? Gmail oblivion9999 if interested. #runners
Tweet 17 tomit posted Have maxed out my Mafia members of Facebook. If anyone wants to play or wants to add me here's the link to my FB page. Weblink
Oblivion said Out stashing water for my 20 miles. About 2 hours behind schedule. Feh. #runners
 
dashdrum said Listening to the SXSW 2009 Showcasing Artists download - set 3This is a very eclectic mix of musicI figured it would all be indie.
Tweet 18 tomit posted I will be accepting almost all adds on Facebook for a little while. Weblink
dashdrum said RT @mjasay: “Facebook is for those you used to know, Twitter is for those who you will know.”(via Aaron Asay)
 
rabbibob said @UberSchwarz If it works, you'll get a kick out of what I'm going to do with it.
Tweet 19 thurrott said Good morning. I emerge from my cave this morning with a few chapters done ("Windows 7 Secrets") but hundreds of screenshots yet to take. Joy
rabbibob said @UberSchwarz I'm accessing info the array supplied from an existing Perl module (Net::Twitter). I hadn't seen [Array[Array{Hash}]] before.
 
Oblivion said Forerunner 305 fixed. Needed to delete all Courses on unit and resend. Had previously seen a "Database Error" on the unit. #runners
Tweet 20 Oblivion said Should shave my head today. Sunburn dictates otherwise.
Oblivion said Bah! Getting delayed on my 20 miles because my Forerunner 305 will not load the Course I made for it after I tweaked it a bit. #runners
 
marcapitman said Oh joy! Girl Scouts were selling cookies at the supermarket today! I DO get my fix this year!
Tweet 21 adamcurry said Done with accountants. Astoundingly simple. Turns out pysical gold makes life easier in that regard smile
Smorgy said Off to bed... talking in the dark about travel memories
 
UberSchwarz said Trying to research White Balance.
Tweet 22 JohnCleese posted Just had a web minion add me to the Weblink  twitter directory under: #British #Actor #Writer
dashdrum said I think I'm the youngest person here - except for the staff.
 
UberSchwarz said @rabbibob what's the purpose of the array?
Tweet 23 Shultzman said woke up to sunny skies and the sound of birds singingI LOVE SPRING!!!
rabbibob said Implemented.  my $location=$timeline[0][0]{'user'}{'location'};  Now off to start my morning.
 
chrismarquardt said Evil Eye  http://twitpic.com/2b5m0
Tweet 24 merlyn said completely off oxycontin; didn't even take any ibuprofen between 9pm and 6am. hardest part has passed. tomorrow: bandages off!
rabbibob said Perlmonks are great (re: a helpful lot).  I have notes now and will look at the AoAoH later.
CPANr said Getopt::Long - Parse long and short command line options in a convenient way. http://tinyurl.com/q4mjc
cc_chapman said Up and at them. Busy morning of Lacrosse and Drama.
cc_chapman said 100 liters of H2O donated for every follower of @WorldH2O @chlorine & Facebook group member http://is.gd/oeen #WWD09 #worldwaterday
DucSloerie said Is this Twitter working?
rabbibob said Befuddled by the array type I'm currently staring at and how to handle it correctly vs building a crescent hammer to deal with it.
Smorgy said Havent tweeted much today.. had a nice lazy day with @fortheinsane
rabbibob said @uberschwarz @draithon short HDR bit discussed on TFTTF #346
drewdomkus said unless some sort of miracle happens, looks like all the important files on the iRiver are really gone. total fail on my part. i'm so pissed.
merlyn said Almost gave up on #Dollhouse, but tonight's episode drew me in. Deep conspiracies, nice. Send me more, quickly!
jluscious said Another crazy night. Counting down the hours to my weened!
YvoLaLu said It says "Von Iva. Period."- Anna pointing @yvolalu lack of a bra
drewdomkus said i stupidly and accidentally formatted an iRiver IFP-790. if anyone knows of any magical way to recover the files, please get in touch. #fb
closedblueyes said "There are three flowers in the vase and one of them is green." Holy shit, tonight's Dollhouse was awesome.
rosicrux said first real bbq of the seasonBison steak, shrimp, zucchini, and fresh corn on thje cob all on the grill to kick it off
jessicamullen said hanging out w @adranchjason + @cynthiaisgr8  about 2 record... + got today's ep up finally! http://thepopularpodcast.com/?p=517
fortheinsane said @tomit 6 seasons, awesome, so depressing but awesome
tomit said @fortheinsane I think I want to try and watch that show now that we don't have BSG. How many seasons did it have?
darthvader said Trust me, it's *not* impossible, just highly improbable. Speaking of improbable, as a boy, I built C-3PO.
fortheinsane said Watching six feet under
wootsellout said $69.99 : Wizcom Quictionary 2 Premium Scanning Pen http://sellout.woot.com
woot said $19.99 : Tornado USB Data Transfer with PC Eraser http://www.woot.com
darthvader said If you're stinging from the BSG spoilers on Twitter, maybe this will take some of the edge off - I am Luke's father.
marcapitman said RT @starfocus: “"Did you sleep well?" "No, I made a couple of mistakes”- Stephen Wright : Hope everyone sleeps well! :)
toyotaboy said @arnell: are you trying to get @comcastcares attention?
tomit said Thinking that I am just going to start using my blog even though I'm still working on it. May push me to work faster.
ambermacarthur said RT @chrisdick - Twitter is like cable, so many channels and nothing on! (Yeah, but like cable, not ready to cut the cord just in case - AM)
DAVEfromNY said Lee went to take a bath and there was no hot water. Turns out our boiler bit the dust :-( Gonna be a cold night and an expen$ive weekend!
Arnell said Comcast is out AGAIN!!!  Man they suck!!!
MuyLani said ah! look at the time. gotta get something to eat before I pick up Olie from her Granny's
closedblueyes said @MuyLani I've gone to a sushi place alone before. It's not terrible, but definitely not as fun as going with friends.
drewdomkus said got word that our new shirts will be ready on tuesday, then i'll process them for delivery. sorry for the delay! http://minionarmy.com
MuyLani said @closedblueyes yes! sushi sounds delicious right now but it's no fun to eat alone :-(
marcapitman said Check out this Twitter map http://bit.ly/12fdG THIS is what I wished LinkedIn offered.
closedblueyes said STARVING for sushi.
marcapitman said RT @kanter: @sproutbuilder @carnet is on my map at about 2 PM right above @dnwallace and next to @marcapitman http://is.gd/ogvl
marcapitman said New personal blog: Out of the mouth of babes: This morning, I told job Caleb to stop talking. .. http://poprl.com/jVF
jessicamullen said change of plans - i now cordially invite you to buy me a drink at longbranch tonight.
justinrussell said Funny how a fairly small traffic change still costs a million bucks to complete. It's a good change, though. http://is.gd/ohcI
jntracks said Last tweet was screaming audience from my view. Not enough lite but they got excited about net fame
LeoLaporte said The side-effects for screening are minimal. The PSA is a blood test that can be done along with other screenings... re: http://ff.im/1Davu
jluscious said Good for you Enzo...dominate that ducky!!! http://twitpic.com/2ateu
jessicamullen said lol anyone want to go to cover 3 with us for dinner and drinks? http://www.yelp.com/biz/cover-3-austin
gentryart said Added more paintings to the site, gentryart.us check out the profile dashdrum wrote for me!
jluscious said Can't seem to shake the tiredness lately... Just exhausted
dashdrum said Can't fit my thought into 140 characters, so forget it!


</pre>
</pre>
=Fetch Code=
=Fetch Code=
<pre>
<pre>
Line 124: Line 87:


my $twit = Net::Twitter->new({username=>"user", password=>"password" });
my $twit = Net::Twitter->new({username=>"user", password=>"password" });
my $last_id=1363944982;  #work this out later
my $last_id=1366625646;  #prime the last_id for first run
 
if (-e "bookmark.txt")  #get the last_id
    {
    open BOOKMARK, "<", "bookmark.txt" or die $!;
    while (<BOOKMARK>)
        {
        if ($_>$last_id)  #keep track of the most recent update to dump to bookmark later
            {
            $last_id=$_;
            }
        }
    close BOOKMARK;
    }
 
my @timeline = $twit->friends_timeline({count => 200, since_id => $last_id });
my @timeline = $twit->friends_timeline({count => 200, since_id => $last_id });
my $timeline;
my $timeline;
my $i;
my $i=0;


while ($timeline[0][$i])
while ($timeline[0][$i])
     {
     {
     my $location=$timeline[0][$i]{'user'}{'location'};
    $i++;
     my $screenname=$timeline[0][$i]{'user'}{'screen_name'};
    }
     my $msg=$timeline[0][$i]{'text'};
 
print "There have been $i tweets since the last update\n\n";
 
my $counter=1;
 
    $i--;
 
for my $textout (reverse 1 .. $i)
    {
 
     my $location=$timeline[0][$textout]{'user'}{'location'};
    my $id=$timeline[0][$textout]{'id'};
   
    if ($id>$last_id)  #keep track of the most recent update to dump to bookmark later
        {
        $last_id=$id;
        }
     my $screenname=$timeline[0][$textout]{'user'}{'screen_name'};
     my $msg=$timeline[0][$textout]{'text'};
   
    my $updatetype;
   
    if ($msg=~m/http/)
        {
        $updatetype="posted";
        }
    else
        {$updatetype="said"}
   
      
      
     $msg=~s/https?:\/\/([-\w\.]+)+(:\d+)?(\/([\w\/_\.]*(\?\S+)?)?)?/link /g;
     # Need to develop a scrubber routine that takes an input file
      
      
     print "$screenname said $msg\n";
    $msg=~s/https?:\/\/([-\w\.]+)+(:\d+)?(\/([\w\/_\.]*(\?\S+)?)?)?/Weblink /g;
     $i++;
    $msg=~s/RT/retweet/g;
    $msg=~s/\.\./\./g;
    $msg=~s/:\)/smile/g;
    $msg=~s/;\)/wink/g;
    $msg=~s/:\(/frown/g;
   
    # Do I want a file list of people to NOT put to the output?   
     print "Tweet $counter $screenname $updatetype $msg\n\n";  
     $counter++;
     }
     }
open BOOKMARK, ">", "bookmark.txt" or die $!;  #set the last_id for the future
print BOOKMARK "$last_id";
close BOOKMARK;
</pre>
</pre>



Latest revision as of 11:30, 24 March 2009

I have no idea if this will go anywhere and the concept came from listening to Adam Curry on No Agenda.

Idea

Premise: parse Twitter feed using Perl, utilize Text2Wav (Festival) to convert to a wav file and then LAME encode the result to mp3.

text2wave -F 16000 htt.txt -o test2.wav
lame -s 8 test2.wav test2.mp3

Audio Result

<flashmp3>http://www.rabbibob.com/cstrike/20090324_twitter.mp3%7Cautostart=no%7Cloop=no%7CfirstTrackName=Twitter2Mp3 Example</flashmp3>

Text

  • implemented 'bookmark' - records id from update to file
    • read in bookmark and start from that marker (200 max)
  • replaced http urls with 'weblink'
  • reduced multiple .. to .
  • replaced minor emoticons with words
  • reversed tweet order (now reads oldest -> newest)
  • reports # of tweets since last update (200 max)
  • added extra line break - helps text2wav 'breath' between tweets
There have been 200 tweets since the last update

Tweet 1 pralphman said Checking out some film trailers on apple.com Seems there are some fairly good comedies coming in the next few months.

Tweet 2 Oblivion said Crap. Yesterday was the first day in 42 that I missed doing at least the body test on the #WiiFit. Foo, its disappointed with me.

Tweet 3 dashdrum said @toyotaboy yeah, but my co-workers don't know they aren't clean :-)

Tweet 4 dashdrum said Hot chocolate and pop-tarts.  Breakfast of champions!

Tweet 5 fortheinsane said @dashdrum what flavour pop tarts?

Tweet 6 draithon said @UberSchwarz u may feel like cattle when they herd u on, but hey for the prices they charge I'm not complaining, ok I want more destinations

Tweet 7 dashdrum said @fortheinsane Brown sugar cinnamon.  The best one!

Tweet 8 draithon said I'm overly concerned about a proposal that could cap a persons compensation for their job The interpretations of that law could be disaster

Tweet 9 draithon said America was built on hard work.  If you cap wages, where is motivation going to be.  Hate to say it but greed is a great motivator.

Tweet 10 draithon said I'm not in finance, but I can see this proposal being interpreted & construed against other jobs.  Ultimately hurting the middle class.

Tweet 11 draithon said I usually try to stay away from politics, but the last 18 months - 2 years I have been overly studying & paying better attention to them.

Tweet 12 draithon said lol

Tweet 13 draithon said Dam @twitterfox is getting aggravating.  It submitted the tweet before I was ready to.  Just another thing for this monday I guess

Tweet 14 adamcurry said Preferred format for promos is mp3

Tweet 15 marcapitman posted Great Foxtrot cartoon on Twitter and parenting Weblink  Props to @trwl

Tweet 16 cc_chapman posted SXSW Was a Success For Me -  Weblink 

Tweet 17 tomit posted Have maxed out my Mafia members of Facebook. If anyone wants to play or wants to add me here's the link to my FB page. Weblink 

Tweet 18 tomit posted I will be accepting almost all adds on Facebook for a little while. Weblink 

Tweet 19 thurrott said Good morning. I emerge from my cave this morning with a few chapters done ("Windows 7 Secrets") but hundreds of screenshots yet to take. Joy

Tweet 20 Oblivion said Should shave my head today. Sunburn dictates otherwise.

Tweet 21 adamcurry said Done with accountants. Astoundingly simple. Turns out pysical gold makes life easier in that regard smile

Tweet 22 JohnCleese posted Just had a web minion add me to the Weblink  twitter directory under:  #British #Actor #Writer

Tweet 23 Shultzman said woke up to sunny skies and the sound of birds singing.  I LOVE SPRING!!!

Tweet 24 merlyn said completely off oxycontin; didn't even take any ibuprofen between 9pm and 6am.  hardest part has passed. tomorrow: bandages off!

Fetch Code

#!/usr/bin/perl
use strict;
use warnings;
use Net::Twitter;

my $twit = Net::Twitter->new({username=>"user", password=>"password" });
my $last_id=1366625646;  #prime the last_id for first run

if (-e "bookmark.txt")   #get the last_id
    {
    open BOOKMARK, "<", "bookmark.txt" or die $!;
    while (<BOOKMARK>)
        {
        if ($_>$last_id)   #keep track of the most recent update to dump to bookmark later
            {
            $last_id=$_;
            }
        }
    close BOOKMARK;
    } 

my @timeline = $twit->friends_timeline({count => 200, since_id => $last_id });
my $timeline;
my $i=0;

while ($timeline[0][$i])
    {
    $i++;
    }

print "There have been $i tweets since the last update\n\n";

my $counter=1;

    $i--;

for my $textout (reverse 1 .. $i)
    {
   
    my $location=$timeline[0][$textout]{'user'}{'location'};
    my $id=$timeline[0][$textout]{'id'};
    
    if ($id>$last_id)   #keep track of the most recent update to dump to bookmark later
        {
        $last_id=$id;
        }
    my $screenname=$timeline[0][$textout]{'user'}{'screen_name'};
    my $msg=$timeline[0][$textout]{'text'};
    
    my $updatetype;
    
    if ($msg=~m/http/)
        {
        $updatetype="posted";
        }
    else
        {$updatetype="said"}
    
    
    # Need to develop a scrubber routine that takes an input file
    
    $msg=~s/https?:\/\/([-\w\.]+)+(:\d+)?(\/([\w\/_\.]*(\?\S+)?)?)?/Weblink /g;
    $msg=~s/RT/retweet/g;
    $msg=~s/\.\./\./g;
    $msg=~s/:\)/smile/g;
    $msg=~s/;\)/wink/g;
    $msg=~s/:\(/frown/g;
    
    # Do I want a file list of people to NOT put to the output?    
    print "Tweet $counter $screenname $updatetype $msg\n\n";    
    $counter++;
    }



open BOOKMARK, ">", "bookmark.txt" or die $!;  #set the last_id for the future
print BOOKMARK "$last_id";
close BOOKMARK;

Code Notes

$msg=~s/https?:\/\/([-\w\.]+)+(:\d+)?(\/([\w\/_\.]*(\?\S+)?)?)?/link /g;

Fetch Result

results in (chopped)

$VAR1 = [
          [
            {
              'source' => 'web',
              'favorited' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
              'truncated' => $VAR1->[0][0]{'favorited'},
              'created_at' => 'Sat Mar 21 10:27:17 +0000 2009',
              'text' => '@uberschwarz @draithon short HDR bit discussed on TFTTF #346',
              'user' => {
                          'location' => 'Behind you...',
                          'followers_count' => 99,
                          'profile_image_url' => 'http://s3.amazonaws.com/twitter_production/profile_images/105270932/3371878294_7e0723920e_normal.jpg',
                          'protected' => $VAR1->[0][0]{'favorited'},
                          'name' => 'Rabbi Bob',
                          'url' => 'http://www.rabbibob.com/',
                          'id' => 2955291,
                          'description' => 'degradable',
                          'screen_name' => 'rabbibob'
                        },
              'in_reply_to_user_id' => 3708131,
              'id' => 1365343447,
              'in_reply_to_status_id' => undef,
              'in_reply_to_screen_name' => 'UberSchwarz'
            }
      ]
]