Actions

Rclone Pictures to Google Drive

From Rabbi Blog

Revision as of 15:32, 16 September 2020 by Rabbi Bob (talk | contribs) (→‎rclone)

Move Files

Perl

  • Purpose:
    • Read source dir
    • Rename files and move to a Year\Month\Day structure
use strict;
use warnings;
use English;
use File::Basename qw( fileparse );
use File::Path qw( make_path );
use File::Spec;
use File::Copy;

my $dir = '/home/somedir/checkmail';
my $destination ='/home/somedir/securitycameras/'; #softlink
foreach my $fp (glob("$dir/*.jpg")) {
#  printf "%s\n", $fp;

print "_______________________________________\n";
print "Source File: $fp\n";

my $filename_year="";
my $filename_month="";
my $filename_day="";
my $filename_camera="";
my $filename_time="";
my $filename_extension="";
my $output_filename="";
#04_20190430185212.jpg

if ( $fp =~ /(\d\d)_(\d{4})(\d{2})(\d{2})(.*)(.jpg)/ ) {
#print "$1\n";
$filename_year=$2;
$filename_month=$3;
$filename_day=$4;
$filename_camera=$1;
$filename_time=$5;
$filename_extension=$6;
$output_filename=$filename_year.$filename_month.$filename_day.'_'.$filename_time.'_'.$filename_camera.$filename_extension;
print "Output File: $output_filename\n";
}
### Check for destination directory - YEAR
my $dest_year = $destination.$filename_year;
print "Dest Year: $dest_year\n";

### Check for destination directory - MONTH
my $dest_month = $dest_year.'/'.$filename_month;
print "Dest Month: $dest_month\n"; 

### Check for destination directory - DAY
my $dest_day = $dest_month.'/'.$filename_day;
print "Dest Day: $dest_day\n";

### Dest File

my $dest_file = $dest_day.'/'.$output_filename;
print "Dest File: $dest_file\n";

############# Create Dirs ################

	if ( !-d $dest_year ) 
		{
		print "Creating $dest_year\n";
    		make_path $dest_year or die "Failed to create path: $dest_year";
		}
	if ( !-d $dest_month ) 
		{
		print "Creating $dest_month\n";	
    		make_path $dest_month or die "Failed to create path: $dest_month";
		}
	if ( !-d $dest_day ) 
		{
		print "Creating $dest_day\n";
    		make_path $dest_day or die "Failed to create path: $dest_day";
		}

	#### MOVE FILES #####
	move ($fp,$dest_file) or die "The move operation failed: $!";
	}

rclone

  • Install rclone
  • Execute rclone config from the account that will be using it
    • N(ew)
    • Name: <NameofRcloneProject>
    • Find the Google Drive entry (13):
    • Client_ID: enter (default)
    • Client_Secret: enter (default)
    • Scope: 1 (we need to look into 3)
      • Assumption: create a folder and get the folder ID for next line
    • Root folder ID: Enter
    • Service Account: Enter
    • Edit Advanced config: n
    • Use autoconfig? N (headless)
      • Copy line to browser that is logged into the Google account to be used
      • Grant rclone access
      • copy the code
    • Enter Verifcation Code: paste the code
    • Configure this as a team drive? N
    • Select: Yes this is OK

Example rclone lines

copy

rclone copy --update --verbose --transfers 30 --checkers 8 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s "/home/someuser/dropbox/sorted" "NameofRcloneProject:NameofGoogleDriveFolder"

sync

rclone sync /home/someuser/dropbox/sorted NameofRcloneProject[1]:GoogleDriveFolder[2] --transfers=4 --checkers=2 --tpslimit=2 --drive-chunk-size=1M --bwlimit 4M

runscript.sh

  • Purpose:
    • make sure rclone isn't running
    • fetchmail, then check (if mail), pull out mime objects
    • run the perl to coordinate
    • run rclone to sync

Contents

if ps ax | grep -v grep | grep -v grep | grep rclone > /dev/null
then
    echo "RCLONE is running, let's not bind the system up"
else
    echo "RCLONE is not running"
fetchmail
cp /var/spool/mail/somedir/home/somedir/
echo 'd *' | mail -N
ripmime -i somedir -d /home/somedir/checkmail
rm /home/somedir/checkmail/text*
perl /home/somedir/movefiles.pl
# rclone copy /home/somedir/securitycameras rclone:securitycameras --transfers=40 --checkers=15 --tpslimit=10 --drive-chunk-size=1M --bwlimit 4M
rclone sync /home/somedir/securitycameras rclone:securitycameras --transfers=4 --checkers=2 --tpslimit=2 --drive-chunk-size=1M --bwlimit 4M
exit 0
fi
  1. Where construction is the Name you gave this process to during rcloneconfig
  2. where Construction is the name of the Google Drive folder