How do I lock read/write to MySQL tables so that I can select and then insert without other programs reading/writing to the database?
You can lock tables using the MySQL LOCK TABLES command like this:
LOCK TABLES tablename WRITE;# Do other queries hereUNLOCK TABLES;See:
Well, table locks are one way to deal with that; but this makes parallel requests impossible. If the table is InnoDB you could force a row lock instead, using SELECT ... FOR UPDATE within a transaction.
BEGIN;SELECT ... FROM your_table WHERE domainname = ... FOR UPDATE# do whatever you have to doCOMMIT;Please note that you will need an index on domainname (or whatever column you use in the WHERE-clause) for this to work, but this makes sense in general and I assume you will have that anyway.
You probably don't want to lock the table. If you do that you'll have to worry about trapping errors when the other crawlers try to write to the database - which is what you were thinking when you said "...terribly complex and relies on many other things."
Instead you should probably wrap the group of queries in a MySQL transaction (see http://dev.mysql.com/doc/refman/5.0/en/commit.html) like this:
START TRANSACTION;SELECT @URL:=url FROM tablewiththeurls WHERE uncrawled=1 ORDER BY somecriterion LIMIT 1;INSERT INTO loggingtable SET url=@URL;COMMIT;Or something close to that.
[edit] I just realized - you could probably do everything you need in a single query and not even have to worry about transactions. Something like this:
INSERT INTO loggingtable (url) SELECT url FROM tablewithurls u LEFT JOIN loggingtable l ON l.url=t.url WHERE {some criterion used to pick the url to work on} AND l.url IS NULL.