Saturday, April 22, 2006

An anti-spam solution

My preferred solution to the spam problem is "Store on server" email. When an email is sent, only the sender information, subject, and location of the full email would be sent. When this header hits the recipient's mailbox, it is checked against a whitelist, and if it's there the email is downloaded (from the location given) immediately. If it doesn't appear on the whitelist, it is then checked against a blacklist, and if it appears there it is deleted without confirmation.

If it isn't on either list, the originating server has to wait for approval by the recipient before the complete email is sent. This would ensure four things: the recipient expresses interest based on the subject and who it's from, the incoming mail server is not spam flooded by thousands of huge emails at once, the sender gives a valid location for the completed email, and the server is still accessible when the recipient asks for it. This prevents bogus return addresses (since there would be no way to see if a response was received) and IP hopping to keep ahead of spam blockers. It would also move much of the network burden from spam from people who receive emails to the people who send them.

I received a response to my suggestion on Slashdot, so I wrote a response (I think the quoted text is "fair use" to show what I'm replying to):

The one giant problem that your idea (and others like it) fail to address is non-support for bulk sending. One of my clients regularly sends about 60,000 copies of his monthly newsletter to opt-in customers.

If it's opt-in, the return address should already be in the whitelist. If it isn't, then the full message would only be downloaded if a person expressed interest in it.

The current system allows him to spool out mail at a pace his system can handle. Your system encourages his server to ignite at 8:15 AM whenever all his recipients get to work, check their mail, and simultaneously attempt to download the message.

This assumes that 60,000 people forget to whitelist the email, yet want to download it simultaneously. It is trivial to set up a filter to whitelist all email addresses you write to (and a simple "respond to the confirmation email" would work for that), so if it's truly "opt in" and not "oops, I forgot to uncheck a hidden box," the email would be downloaded in its entirety as soon as the header hits the mailbox.

In addition, one bit of information that can be included in the subject/sender/header info is an email expiration. If you get hammered every day at 8:15 am because many of your opt-in customers have no clue what they are doing, just set the expiration to 8 am. They would see a message that says "not on whitelist/expired" when attempting to download the message, and they can choose at that point to whitelist, blacklist, or keep current settings for future communications.

Sunday, April 02, 2006

Thoughts on Education, AKA Educational Triage

Q: How is it possible to measure teacher ability?

A: By comparing student progress. If a class makes significantly more progress than average, then I would say the teacher is effective. If a class makes significantly less progress than the average, the teacher is not effective. In other words, if a class goes from fourth to sixth grade reading level in one year, the teacher is probably a good one, but if the class goes from grade 6 to grade 6.5, then the teacher is probably not as good, even though the final level is higher (BTW, this would help alleviate some of the complaints concerning teachers in remedial classes, since this would measure how much is gained rather than an absolute score).

Q: When there just aren't enough teachers in raw numbers, how can you select for talent? If a business needs 10 engineers and can only find 8 competent ones, they still hire 10 people. So you have either 2 non-engineers trying to learn how to do it or 2 incompetent engineers with the other 8 trying to carry their weight.

A: Educational triage. If you don't have enough resources to help everyone, you shift the resources where they will do the most good. In your example, the best engineers would be placed on the most critical projects, and the incompetent ones would be put where they do the least harm. It sounds cold-blooded and calculating, but so does medical triage, and if we can't aim for a perfect outcome, we have to try for the best possible outcome under the circumstances.

Of course, there are two different viewpoints of the "best possible outcome." We can try to maximize overall achievement, even if some students fail, or we can try to minimize overall differences (No child left behind, anyone?), which results in varying degrees of mediocrity. If you want to emphasize overall accomplishment -- which I would suggest -- match the best teachers with the students that have most potential for learning. If you want to minimize differences, reverse the order.

A short note about immigration

I posted this on Yahoo, but I like to keep copies of what I write in one place, in case I have to edit, update, or defend my statements. Here are my thoughts on illegal immigration:

Immigrants are welcome as long as they want to become Americans rather than turning America into whatever country they fled from. That means learning English, leaving the Mexican flags at home, and supporting yourself without government assistance.

For those claiming that the U.S. is heartless -- apparently, we are far less heartless than Mexico, otherwise people wouldn't be fleeing that country to settle in a place with a different language and customs. Let's see illegal aliens protest the awful hellhole they came from instead of the place they are escaping to.

I personally don't think this problem will be solved by legislation -- look at the amnesty Reagan gave, in the mistaken belief that it was a one-time deal, and people south of the border would suddenly obey U.S. laws. It probably won't be solved by a Great Wall on our border, either, not because of flawed analogies to the Berlin Wall (which was designed to keep people *in*, not *out*), but because people will still go over it, under it, or around it, even if it means building boats and using the ocean.

The only way it will be solved (in my opinion) is if the Mexican government has a string of honest, intelligent leaders -- good luck with that -- or the U.S. annexes Mexico.