Category Archives: Measurement

Transparency in Rome

Here’s my presentation, Transparency as Incentive for Internet Security: Organizational Layers for Reputation, from RIPE 61 in Rome. This presentation summarizes the two previous RIPE Labs papers about proposed new organizational layers and outbound spam ranking experiments.

RIPE-NCC is the oldest of the Regional Internet Registries (RIRs), and RIPE is the deliberately unorganized association of interested parties that meets twice a year and holds discussions online in between. It’s a mix of operations, research, and socializing. Topics range from obscure details of deploying IPv6 to organizational proposals such as what I was talking about. 430 people attended the meeting in Rome, which was quite a few more than the dozen or two of the first RIPE meeting I went to many years ago.

Interesting questions were asked. I may blog some of them.

-jsq

Daniel Karrenberg and RIPE Atlas

Daniel Karrenberg shows an animation related to RIPE Atlas, RIPE’s new active measurement project using USB-powered dongles scattered around the Internet.

Video by jsq at RIPE 61 in Rome, Italy, 15 Nov 2010. His slides, the RIPE Atlas home page, and the conference will put up video of all the talks within about a day.

-jsq

PS: My talk is 11AM Rome time tomorrow, Tuesday 16 Nov.

Outbound Spam Ranking Experiments

Should Uganda Telecom be counted as a Belgian ISP for outbound spam rankings?

Which matters most: history, topology, business headquarters location, or some other criterion?

These are some questions that come up in designing experiments in rolling out a reputation system for outbound spam. More on this in the RIPE Labs article (8 Nov 2010), Internet Reputation Experiments for Better Security.

Such experiments can draw on fifty years of social science research and literature, first crystalized as Social Comparison Theory by Leon Festinger in 1954, that indicate that making personal reputation transparent changes personal behavior. More recent research indicates that the same applies to organizations. Using anti-spam blocklist data, it is possible to make E-Mail Service Provider (ESP) behavior (banks, stores, universities, etc., not just ISPs) in preventing or stopping outbound spam transparent, and this paper is about experiments to see how the resulting reputation actually changes ESP behavior.

-jsq

Organizing the Cloud Against Spam

In RIPE Labs, here’s a paper on Internet Cloud Layers for Economic Incentives for Internet Security by the IIAR Project (I’m the lead author). Anti-spam blocklists and law enforcement are some Internet organizational layers attempting to deal with the plague of spam, so far reaching a standoff where most users don’t see most spam, yet service providers spend large amounts of computing and people resources blocking it.
The root of the ecrime problem is not technology: it is money.
Continue reading →

NANOG: Botnets, DDoS and Ground-Truth

Here at NANOG 50 Craig Labovitz just gave an interesting talk about botnet data derived from Arbor Network customers enabling anonymous data (37 ISPs over last 12 months), of 5,000 events classified by operators.

60% of DDoS attacks are by flooding. Yet most attacks involve few IP addresses; indicates address spoofing.

Slight problem: only 1/4 of customers have enabled anonymous data. “Real goal of this talk is to encourage participation.”

Well-received talk.

-jsq

Quis custodiet ipsos medici?

Internet security is in a position similar to that of safety in the medical industry. Many doctors have an opinion like this one, quoted by Kent Bottles:
“Only 33% of my patients with diabetes have glycated hemoglobin levels that are at goal. Only 44% have cholesterol levels at goal. A measly 26% have blood pressure at goal. All my grades are well below my institution’s targets.” And she says, “I don’t even bother checking the results anymore. I just quietly push the reports under my pile of unread journals, phone messages, insurance forms, and prior authorizations.”

Meanwhile, according to the CDC, 99,000 people die in the U.S. per year because of health-care associated infections. That is equivalent of an airliner crash every day. It’s three times the rate of deaths by automobile accidents.

The basic medical error problems observed by Dennis Quaid when his twin babies almost died due to repeated massive medically-administered overdoses and due to software problems such as ably analysed by Nancy Leveson for the infamous 1980s Therac-25 cancer-radiation device are not in any way unique to computing in medicine. The solutions to those problems are analogous to some of the solutions IT security needs: measurements plus six or seven layers of aggregation, analysis, and distribution.

As Gardiner Harris reported in the New York Times, August 20, 2010, another problem is that intravenous and feeding tubes are not distinguished by shape or color: Continue reading →

What we can learn from the Therac-25

What does Nancy Leveson’s classic analysis of the Therac-25 recommend? (“An Investigation of the Therac-25 Accidents,” by Nancy Leveson, University of Washington and Clark S. Turner, University of California, Irvine, IEEE Computer, Vol. 26, No. 7, July 1993, pp. 18-41.)
“Inadequate Investigation or Followup on Accident Reports. Every company building safety-critical systems should have audit trails and analysis procedures that are applied whenever any hint of a problem is found that might lead to an accident.” p. 47

“Government Oversight and Standards. Once the FDA got involved in the Therac-25, their response was impressive, especially considering how little experience they had with similar problems in computer-controlled medical devices. Since the Therac-25 events, the FDA has moved to improve the reporting system and to augment their procedures and guidelines to include software. The input and pressure from the user group was also important in getting the machine fixed and provides an important lesson to users in other industries.” pp. 48-49

The lesson being that you have to have built-in audit, reporting, transparency, and user visibility for reputation.

Which is exactly what Dennis Quaid is asking for.

Remember, most of those 99,000 deaths a year from medical errors aren’t due to control of complicated therapy equipment: Continue reading →

What about the Therac-25?

Someone suggested that Dennis Quaid should be reminded of the Therac-25 “if he thinks computers will reduce risk without a huge investment in quality, quality assurance and operational analysis.” For readers who may not be familiar with it, the Therac-25 was a Canadian radiation-therapy device of the 1980s that was intended to treat cancer. It had at least six major accidents and caused three fatalities, because of poor software design and development.

Why should anyone assume Dennis Quaid doesn’t know that quality assurance and operational analysis are needed for anything designed or controled by software? The man is a jet pilot, and thus must be aware of such efforts by aircraft manufacturers, airlines, and the FAA. As Quaid points out, we don’t have a major airline crash every day, and we do have the equivalent in deaths from medical errors. Many of which could be fixed by Computerized Physician Order Entry (CPOE).

Or ask the Mayo Clinic: Continue reading →

Trust the Doctor, or Trust the Doctor’s Report Card?

What can be done about the huge medical error fatality problem Dennis Quaid identified when his baby twins were almost killed? Electronic medical records (EMR) are a start. Then as Kent Bottles suggests, let’s use those records to improve physician care:
“Dr. Kim A. Adcock, the radiology chief at Kaiser Permanente Colorado, created a system that misses one-third fewer cancers on mammograms and “has achieved what experts say is nearly as high a level of accuracy as mammography can offer.” At the heart of the program was his willingness to keep score and confront his doctors with their results. He had to fire three radiologists who missed too many cancers, and he had to reassign 8 doctors who were not reading enough films to stay sharp.”
We could use more report cards for physicians, including firing ones with failing grades, and maybe even paying the really good ones more, or at least getting them to teach the others.

-jsq