LITTLEBLACKDOG.COM

Posted: Sat Jan 06, 2007 5:56 pm Post subject: Javascript, image age, hash, etc.

I'm looking for a way to remotely retrieve an image (using javascript), and determine the age of the image, or some way to figure out if its been updated.

If I could get the "creation" date from the image, that would work best

but I'd also be interested in making a short md5 style hash of the image to compare to a stored md5 hash to determine if this image has been altered since it was originally hashed. if changed, then you replace the hash, update the date, and go on. if not, then you leave the original hash, and can determine how old it is by the date of the stored hash. not perfect, but at least workable

anyone seen any scripts like this, or have an idea how one could be done?!?!

Thanks

Posted: Sun Jan 07, 2007 8:03 am Post subject:

Do you have access to the server that the image is on?

Reason that I ask is because I don't know of a way off the top of my head that you can determine the age of an image on a website without having access to the image itself.

Posted: Tue Jan 09, 2007 1:07 am Post subject:

Skookum wrote:
Do you have access to the server that the image is on?

Reason that I ask is because I don't know of a way off the top of my head that you can determine the age of an image on a website without having access to the image itself.

hence the problem

I know in php I could get the size of an image, as well as the height and width.

basically, I just need a way to identify identical pictures on a remote site, even tho they have the same name and location.

even making an md5 hash from every image would work, as long as I personally don't have to load it up in php... (horrors)

tho.. I supposed I could do that once a day as well if I had to... but.. then if a camera (for example) is off when I run my daily script, even if its ONLY off for that moment.. it would never get recorded.

I'm trying to find a way to determine when an image was last updated. get rid of some of those static images that are at least a year old on lwdcams.

and I just can't figure out how to write it.

Posted: Tue Jan 09, 2007 5:30 am Post subject:

Well if you just need the heighth, width and size of the image put into a hash for comparison later on, then IF I actually ever make it to the office today I will write something up.

No gurantees though for this week, I have been the cabling monkey at work lately.

Posted: Tue Jan 09, 2007 2:54 pm Post subject:

I just found a function that maybe able to do everything for you.

imagecolorat

Basically you could download the picture as is, then check the center pixel of the image and compare it with previous samples to determine if it is the same raster color or not.

I will try to write a script for it a little later.

Posted: Tue Jan 09, 2007 3:13 pm Post subject:

fathertyme wrote:
even making an md5 hash from every image would work, as long as I personally don't have to load it up in php... (horrors)

What do you mean by load it up? Does PHP not have cryptographic hash functions available for it?

Personally I think an MD5 (or SHA1 would be even better) would be the best way do it.

Posted: Tue Jan 09, 2007 3:23 pm Post subject:

CheeseMonger The Great wrote:
fathertyme wrote:
even making an md5 hash from every image would work, as long as I personally don't have to load it up in php... (horrors)

What do you mean by load it up? Does PHP not have cryptographic hash functions available for it?

Personally I think an MD5 (or SHA1 would be even better) would be the best way do it.

PHP does have an MD5 and SHA1 function for creating hashes.

Posted: Tue Jan 09, 2007 3:27 pm Post subject:

So what's the problem?

Take the hash of the entire image, note the date the it was taken, if the hash is the same after some delta time, ditch the cam. Exactly like FT described in his first post. It's a perfectly legitimate way to do it...

Also, if it absolutely must be done in Javascript, take a look here: http://www.pajhome.org.uk/crypt/md5/md5src.html

Posted: Tue Jan 09, 2007 4:22 pm Post subject:

CheeseMonger The Great wrote:
fathertyme wrote:
even making an md5 hash from every image would work, as long as I personally don't have to load it up in php... (horrors)

What do you mean by load it up? Does PHP not have cryptographic hash functions available for it?

Personally I think an MD5 (or SHA1 would be even better) would be the best way do it.

As it stands right now, I present a web page to people with external images. I don't have anywhere near enough bandwidth to actually serve those images myself.

But I would still like a way to determine if the images loaded, or are new or not. if I could make an MD5 or SHA1 hash I could use javascript to send that to my server and let the server just check hashes.

that help?

Posted: Wed Jan 10, 2007 12:27 am Post subject:

fathertyme wrote:
CheeseMonger The Great wrote:
fathertyme wrote:
even making an md5 hash from every image would work, as long as I personally don't have to load it up in php... (horrors)

What do you mean by load it up? Does PHP not have cryptographic hash functions available for it?

Personally I think an MD5 (or SHA1 would be even better) would be the best way do it.

As it stands right now, I present a web page to people with external images. I don't have anywhere near enough bandwidth to actually serve those images myself.

But I would still like a way to determine if the images loaded, or are new or not. if I could make an MD5 or SHA1 hash I could use javascript to send that to my server and let the server just check hashes.

that help?

Ah, I understand.

You will still need to download the image onto whatever machine is running the script to take the hash though, no?

Posted: Wed Jan 10, 2007 9:33 am Post subject:

yeah, but using javascript would allow me to do the calculations on the client computer instead of mine Smile

Posted: Wed Jan 10, 2007 10:26 am Post subject:

fathertyme wrote:
yeah, but using javascript would allow me to do the calculations on the client computer instead of mine

Ah, blatant cheekiness and subtle cunning. I like it, but...

In the meantime I've had a better idea. It is very cheap in both bandwidth (no up/downloads) and calcualtion costs (no hashing).

You can issue a HEAD request for the image (from PHP or Javascript) and get the image's last modified date, without actually GETting the image itself. For example, the following request:

Code: Select all

HEAD /templates/lwd/images/phpbb2_logo.gif HTTP/1.1                                                             


Host: www.littleblackdog.com


Connection: close

(You can just type this into a telnet session to www.littleblackdog.com on port 80 to try it. Hit return twice to send the request.)

Gives the following reply:

Code: Select all

HTTP/1.1 200 OK


Date: Wed, 10 Jan 2007 18:19:18 GMT


Server: Apache Webserver


Last-Modified: Sun, 29 Oct 2006 07:26:50 GMT


ETag: "4180f2-8f7-454457ba"


Accept-Ranges: bytes


Content-Length: 2295


Connection: close


Content-Type: image/gif

All you have to is parse out the date the image was last modified from the reply header.

Is this helpful at all?

Edit: Also, I reckon a modification date is more reliable than a creation date. A program may choose to truncate a file to zero length and write it rather than delete and recreate it, which would mean that the creation date would not change when it is updated.

Posted: Wed Jan 10, 2007 10:43 am Post subject:

thats a pretty good idea.. not quite sure how to code it, but I like it Wink

Posted: Wed Jan 10, 2007 11:44 am Post subject:

Apparently, you can also make the request conditional:

Quote:
The If-Modified-Since request-header field is used with a method to make it conditional: if the requested variant has not been modified since the time specified in this field, an entity will not be returned from the server; instead, a 304 (not modified) response will be returned without any message-body.

I might a little script to test some of these things out. I'll post it here if I create something useful.

Posted: Fri Jan 12, 2007 5:14 pm Post subject:

I had a couple of hours spare this evening, so I've had a play with this. Now, I don't have PHP set up here so this here proof of concept is in Perl. But it does seem to do what you want, which is to say that it can determine if a file is new without actually downloading it. (Where "new" in this case means "was modified after some arbitrary point in the past.")

I just hard-coded a date to compare against, but it would of course be trivial to fetch that from a DB and store an updated one. I'm sure there must be some facility to do something very similar in PHP regarding sending an HTTP request. A research exercise for the reader. Wink

Source code:

Code: Select all

#!/usr/bin/perrrrllllahemstupidlamenessfilter





use strict;


use LWP::UserAgent;


use HTTP::Date; # for the date string to time function





my $ua = LWP::UserAgent->new;





print "Content-Type: text/plain\n\n";





# default values


my $url1 = 'http://www.matbooth.co.uk/';


my $url2 = 'http://www.littleblackdog.com/templates/lwd/images/phpbb2_logo.gif';


my $date = 'Mon, 1 Jan 2007 00:00:00 GMT';





send_req($url1);


send_req($url2);





# send the request, takes a url


sub send_req


{


   my $url = shift;





   # create request and get response


   print "\nSending a HEAD request to $url...\n";


   my $req = HTTP::Request->new(HEAD => $url);


   my $res = $ua->request($req);





   # check the outcome


   if ($res->is_success) {


      print "Request successful, the server sent the following response:\n\n";


      foreach my $header ($res->header_field_names) {


         print "$header: " . $res->header($header) . "\n";


      }


      if (str2time($res->header('Last-Modified')) > str2time($date)) {


         print "\nThe requested file has changed since $date.\n\n";


      }


      else {


         print "\nThe requested file has NOT changed since $date.\n\n";


      }


   }


   else {


      print "Request failed, the server responded with status " . $res->status_line . ".\n\n";


   }


}