Friday, 06 November
22:37
PNSQC Wrap-Up: The Feedback [Marlena's Blog]
This post is about feedback I received regarding my presentation at PNSQC and my other presentations at Adobe and Microsoft.
Before I mention my feedback, though, I would like to thank Lanette Creamer and Alan Page. They welcomed me to their companies with such hospitality and respect. These are both extremely busy people who took significant amounts of time out of their day to show me around and make me feel at home. This kind of attention is extremely motivating for me, and shows that they lead from the heart.
On to feedback:
What I didn’t know about PNSQC prior to arrival is that they have a feedback system. Whenever an audience member leaves a talk, they can submit a feedback card. The cards are green, yellow and red. Green indicates a good talk with no problems. Yellow means that the presentation was pretty good and may have had some issues. Red means that the presentation was a waste of time.
What did I think of my presentation? I was a nervous wreck, which is extremely out-of-character for me in speaking to people. The moment I started speaking I forgot to breathe. Once that happens, it takes me a good 5 to 10 minutes to recover. My content was challenging to present because I didn’t have bullet points in my slides. In retrospect, I would have spent more time memorizing an outline since I wasn’t going to have that type of aid on screen. My presentations at Adobe and Microsoft were far better.
Here’s a bar graph of the cards I received.
When people turn in cards, they can also write comments on the cards.
Comments on green cards:
1. Good talk: a) should all metrics be visualized? b) need clear goals or don’t bother c)green card mostly for thought production.
2. Great content. Practice your presentation skills
Comments on yellow cards:
1. Too slow
2. Great potential, but message didn’t come across clearly. How to read new visual organizations of data is very important instruction to being able to interpret the pics.
Comments on red card:
Visualization shown was a hard way to view list of failures and counts grouped by component.
I also had some comments from my talks at Microsoft and Adobe. These are comments I remember and are as accurate as possible. Since this post is my bucket for feedback, I’m not answering criticism in this particular post. Don’t worry…you’ll be hearing PLENTY from me later, but this post is about what I heard, not what I think. For anyone who’s been waiting to add their $2, the comments for this post would be a good place.
Alas…the criticism:
The process I’m using is static, although at Microsoft there is more of an interest in real time data.
The purpose of these visualizations is unclear.
Someone on twitter mentioned that presentation started slowly for him.
There were two guys in the back who seemed to have read some stuff about visualization and both were highly critical of my work. I didn’t get to talk to them afterward which is a shame because the only way for me to refine my work is by hearing good criticism. If either of you two dudes is reading, please get in touch.
My example with Parallel Sets is using fake data
None of my examples is being used IRL.
There are not enough configuration options
Some took issue with the title of my paper and presentation. “Where’s the quality? I’m not seeing it.”
Why is the white on the treemap of tests at all?
where is the zooming?
Someone did find a bug in the strip treemap algorithm. This is an internationalization problem. A very nice man was trying to understand the order of the strip treemap. The items in this type of layout are ordered horizontally, and this man was concerned with the vertical ordering. He was Asian and I know that Asian writing is vertically ordered and not horizontally ordered. There is currently no strip layout algorithm I know of that will order the items vertically. Since we live in a global economy, I’m considering this a bug of the strip layout as currently implemented.
Americanface, Episodes 9-12 [Tyee - Home]
All this week's episodes rolled into one.
Regulators agree on proposed global privacy standards [Canadian Privacy Law Blog]
Privacy regulators from around the world, meeting this week in Madrid, have agreed to a framework for international privacy standards. From what I've been able to glean, it would all be consistent with what we currently have in Canada under PIPEDA.
AFP: Experts agree on proposed global privacy standardsExperts agree on proposed global privacy standards
MADRID — Experts from 50 nations meeting in Madrid have reached a draft agreement on international standards for the protection of privacy and personal data, participants said Friday.
Under the proposed standards, data may only be processed after obtaining the "free, unambiguous and informed consent" of the data subjects and it should be deleted when it is no longer necessary for the purposes for which it was gathered.
Data collectors must identify themselves, state in clear language the purpose of the data processing and the recipients of the gathered data.
International transfers of personal data may only be carried out to a country which "affords, as a minimum, the level of protection provided for in the document," according to the proposed standards, agreed by representatives from privacy protection agencies.
"This agreement was reached with the active participation and support of civil society and industry," the head of the Spanish Data Protection Agency, Artemi Rallo Lombarte, said at the end of the three-day gathering.
Participants hope the draft international standards will serve as the basis for a universal, binding legal instrument on data protection. But several cautioned that this is still a long way off given the different rules around the world.
"We have jumped over a first step but we have a long road, a very long road, ahead to arrive at a common, restricting legal framework," said the president of France's CNIL data protection agency, Alex Turk.
Over 1,000 participants from around the world took part in the 31st International Conference of Data Protection and Privacy which is billed as the world's largest forum dedicated to privacy.
US Homeland Security Secretary Janet Napolitano and representatives from key Internet firms like Google and Facebook were among those who took part in the event, which was organized by the Spanish Data Protection Agency.
The next such conference is scheduled for October 2010 in Jerusalem. Previous gatherings have taken place in Strasbourg, Hong Kong, Sydney and Montreal.
Thanks to Alex Cameron for the pointer.
The Doghouse: ADE 651 [Schneier on Security]
A divining rod to find explosives in Iraq:
ATSC’s promotional material claims that its device can find guns, ammunition, drugs, truffles, human bodies and even contraband ivory at distances up to a kilometer, underground, through walls, underwater or even from airplanes three miles high. The device works on “electrostatic magnetic ion attraction,” ATSC says.To detect materials, the operator puts an array of plastic-coated cardboard cards with bar codes into a holder connected to the wand by a cable. “It would be laughable,” Colonel Bidlack said, “except someone down the street from you is counting on this to keep bombs off the streets.”
Proponents of the wand often argue that errors stem from the human operator, who they say must be rested, with a steady pulse and body temperature, before using the device.
Then the operator must walk in place a few moments to “charge” the device, since it has no battery or other power source, and walk with the wand at right angles to the body. If there are explosives or drugs to the operator’s left, the wand is supposed to swivel to the operator’s left and point at them.
If, as often happens, no explosives or weapons are found, the police may blame a false positive on other things found in the car, like perfume, air fresheners or gold fillings in the driver’s teeth.
Complete quackery, sold by Cumberland Industries:
Still, the Iraqi government has purchased more than 1,500 of the devices, known as the ADE 651, at costs from $16,500 to $60,000 each. Nearly every police checkpoint, and many Iraqi military checkpoints, have one of the devices, which are now normally used in place of physical inspections of vehicles.
James Randi says:
This Foundation will give you our million-dollar prize upon the successful testing of the ADE651® device. Such test can be performed by anyone, anywhere, under your conditions, by you or by any appointed person or persons, in direct satisfaction of any or all of the provisions laid out above by you.No one will respond to this, because the ADE651® is a useless, quack, device which cannot perform any other function than separating naïve persons from their money. It’s a fake, a scam, a swindle, and a blatant fraud. The manufacturers, distributors, vendors, advertisers, and retailers of the ADE651® device are criminals, liars, and thieves who will ignore this challenge because they know the device, the theory, the described principles of operation, and the technical descriptions given, are nonsense, lies, and fraudulent.
And he quotes from the Cumberland Industries literature (not online, unfortunately):
Ignores All Known Concealment Methods. By programming the detection cards to specifically target a particular substance, (through the proprietary process of electro-static matching of the ionic charge and structure of the substance), the ADE651® will “by-pass” all known attempts to conceal the target substance. It has been shown to penetrate Lead, other metals, concrete, and other matter (including hiding in the body) used in attempts to block the attraction.No Consumables nor Maintenance Contracts Required. Unlike Trace Detectors that require the supply of sample traps, the ADE651® does not utilize any consumables (exceptions include: cotton-gloves and cleanser) thereby reducing the operational costs of the equipment. The equipment is Operator maintained and requires no ongoing maintenance service contracts. It comes with a hardware three year warranty. Since the equipment is powered electro statically, there are no batteries or conventional power supplies to change or maintain.
One interesting point is that the effectiveness of this device depends strongly on what the bad guys think about its effectiveness. If the bad guys think it works, they have to find someone who is 1) willing to kill himself, and 2) rational enough to keep his cool while being tested by one of these things. I'll bet that the ADE651 makes it harder to recruit suicide bombers.
But what happened to the days when you could buy a divining rod for $100?
11:32
how to surf with a torch [coreman's blog]
Here is the instructional video.
CTV: disappointing Olympic photo gallery [coreman's blog]
So I head out onto the web to look for Olympic torch photos. Most specifically, I want to see this thing about the torch riding a wave up in Tofino. I end up on the CTV BC web-site, and they have this photo gallery - great! I click on the link ...
Window of Opportunity [Abakas]
It's relatively easy to decide to change things. It's even fairly easy to decide what you're going to do differently, generally. In order to be successful, though, you also have to consider when you make the change.
Being a tester, I see a lot of tickets. Some tickets, unfortunately, hang around for a while, and tend to be worked on by multiple people. These wind up with the basic ticket writeup and a series of comments by different people. Particularly when the ticket is a difficult one, there are theories being tried and discarded.
- describe how it relates to the issue as a whole
- describe what the reader is intended to do or take away from the comment
Layton and Donnelly Woo the HST Haters [Tyee - Home]
New Dems' by-election strategy is to pin the much maligned tax on Harper.
This Is Independent Media's Moment [Tyee - Home]
Tomorrow is Media Democracy Day in Vancouver. The potential is huge.
Vancouver's Asian Film Fest Is Nothing to Sneeze at [Tyee - Home]
It showcases Canadian and US filmmakers of Asian descent, and the audience is growing.
Americanface, Episode 12 [Tyee - Home]
Story so far: Surprised by joy on Chestermans Beach, I open a whole new can of worms.
The Curious Case of the Missing Tory [Tyee - Home]
We tried to interview Diana Dilworth, who wants to win the New West-Coquitlam by-election. We really did try.
Karzai's Afghanistan, Last Nail in the West's Coffin [Tyee - Home]
The most powerful empire in history proves to be dumb as a bag of hammers.
Dodging from a Distance [Tyee - Home]
How Cold Cave sidesteps the long shadow of Joy Division.
Holy Flying Digital Dildos! [Tyee - Home]
You might find them offensive, but is it copyright infringement?
Americanface, Episode 11 [Tyee - Home]
Story so far: Six months on Chestermans Beach has me looking at the world in a whole new way.
Americanface, Episode 10 [Tyee - Home]
Story So Far: Turns out, everything I know is wrong.
$6.7 Billion Lopped off Olympic Economic Benefits Projections [Tyee - Home]
And government's new number, $4 billion, is based on a seven-year-old 'best case' scenario that hasn't panned out.
Nova Scotia to probe juror vetting [Canadian Privacy Law Blog]
From today's National Post:
Nova Scotia launches probe into jury vettingShannon Kari, National Post
Published: Thursday, November 05, 2009
The Public Prosecution Service in Nova Scotia is conducting an internal review into whether or not its Crown attorneys have been conducting improper background checks of potential jurors.
The review was prompted by a request for information from the Nova Scotia Criminal Lawyers' Association following a National Post story last month that suggested jury vetting was taking place in the province.
A report issued about the scope of the practice in Ontario by the provincial Privacy Commissioner also cited a senior Crown official in Nova Scotia who said it was common for jury lists to be given to police to do background checks. The information was "generally not shared with the defence," the Nova Scotia official told the Ontario agency. The Public Prosecution Service in Nova Scotia initially suggested the Ontario report was inaccurate. But in a written response to the lawyers' association, the director of public prosecutions in Nova Scotia announced that he had initiated an internal review.
"I anticipate that our review will ultimately result in a policy statement or practice advice being prepared which will be distributed to our Crown attorneys. A copy of that advice piece will be provided to you," wrote Martin Herschorn in the letter dated Oct. 20.
Mr. Herschorn also imposed an interim directive. It states that if a background check is requested, it should only be to see if an individual has been previously sentenced to more than two years in prison, which would make the person ineligible to serve as a juror in Nova Scotia.
A spokeswoman for the prosecution service confirmed yesterday that it is asking its 20 Crown offices whether confidential databases were used to probe potential jurors and if the data was disclosed to the defence. "We hope to have a policy in place by the end of the calendar year," Chris Hansen said.
While he welcomes the review, the president of the Criminal Lawyers' Association said he wants to know more about what happened previously. "Our membership certainly has many more questions," Josh Arnold said. "So far, all the talk has been on a go forward basis."
Dulcie McCallum, the Nova Scotia Freedom of Information and Protection of Privacy review officer, said yesterday that she will wait for the internal review to be completed before deciding whether to launch her own investigation.
The Ontario Privacy Commissioner's report revealed that one in three Crown offices in the province engaged in improper jury vetting in just the past three years.
The Ontario Court of Appeal is hearing its first case on this issue this fall, in which three defendants convicted of murder are seeking a new trial. The Ontario government announced on Oct. 27 that it is amending the Juries Act so that any checks for eligibility will be done by an independent agency and the information will be kept confidential. Mr. Arnold urged the Nova Scotia government to consider similar changes to its Juries Act.
Read more: http://www.nationalpost.com/news/canada/story.html?id=2187075#ixzz0VzXDWvNI
Text of Bill 64, Personal Health Information Act (Nova Scotia) now available [Canadian Privacy Law Blog]
The text of Bill 64, the Personal Health Information Act has now been posted on the Nova Scotia Legislature website.
Personal Health Information Act introduced in Nova Scotia [Canadian Privacy Law Blog]
The Minister of Health for Nova Scotia has today introduced the Personal Health Information Act in the legislature. I'll have a link to the text of the bill tomorrow, but in the meantime you can read the release:
Personal Health Information Legislation Introduced News Releases Government of Nova ScotiaPersonal Health Information Legislation Introduced
Department of Health
November 4, 2009 2:46 PM
Nova Scotian's personal health information would be better managed under proposed legislation introduced today, Nov. 4.
The Personal Health Information Act would provide consistent provincial rules for the management of personal information in health care.
"Patient privacy is a fundamental principle in delivering health care. At the same time, it is important that health care professionals can share information in ways that can improve care," said Health Minister Maureen MacDonald. "This legislation balances these important objectives."
The proposed legislation sets out rules for how health information is collected, used, disclosed, retained and destroyed by the health-care sector in Nova Scotia. It better supports a system that uses electronic as well as paper health records and helps provide a more seamless flow of information.
Specific rules include provisions for privacy breach notification audit reports to track who has had access to electronic health records, and requests for people to access to their health information.
Nova Scotia does not have clear health information legislation. It is governed by a mix of federal and provincial laws, health profession codes, and organizational policies and procedures. Nova Scotia joins eight other provinces who have comprehensive legislation to manage personal health information.
I understand that the legislature session ends shortly, so the Bill will not be debated until the new year. It's also reported that the Department plans to have the Bill come into force in January 2011.
Fear and Overreaction [Schneier on Security]
It's hard work being prey. Watch the birds at a feeder. They're constantly on alert, and will fly away from food -- from easy nutrition -- at the slightest movement or sound. Given that I've never, ever seen a bird plucked from a feeder by a predator, it seems like a whole lot of wasted effort against not very big a threat.
Assessing and reacting to risk is one of the most important things a living creature has to deal with. The amygdala, an ancient part of the brain that first evolved in primitive fishes, has that job. It's what's responsible for the fight-or-flight reflex. Adrenaline in the bloodstream, increased heart rate, increased muscle tension, sweaty palms; that's the amygdala in action. And it works fast, faster than consciousnesses: show someone a snake and their amygdala will react before their conscious brain registers that they're looking at a snake.
Fear motivates all sorts of animal behaviors. Schooling, flocking, and herding are all security measures. Not only is it less likely that any member of the group will be eaten, but each member of the group has to spend less time watching out for predators. Animals as diverse as bumblebees and monkeys both avoid food in areas where predators are common. Different prey species have developed various alarm calls, some surprisingly specific. And some prey species have even evolved to react to the alarms given off by other species.
Evolutionary biologist Randolph Nesse has studied animal defenses, particularly those that seem to be overreactions. These defenses are mostly all-or-nothing; a creature can't do them halfway. Birds flying off, sea cucumbers expelling their stomachs, and vomiting are all examples. Using signal detection theory, Nesse showed that all-or-nothing defenses are expected to have many false alarms. "The smoke detector principle shows that the overresponsiveness of many defenses is an illusion. The defenses appear overresponsive because they are 'inexpensive' compared to the harms they protect against and because errors of too little defense are often more costly than errors of too much defense."
So according to the theory, if flight costs 100 calories, both in flying and lost eating time, and there's a 1 in 100 chance of being eaten if you don't fly away, it's smarter for survival to use up 10,000 calories repeatedly flying at the slightest movement even though there's a 99 percent false alarm rate. Whatever the numbers happen to be for a particular species, it has evolved to get the trade-off right.
This makes sense, until the conditions that the species evolved under change quicker than evolution can react to. Even though there are far fewer predators in the city, birds at my feeder react as if they were in the primal forest. Even birds safe in a zoo's aviary don't realize that the situation has changed.
Humans are both no different and very different. We, too, feel fear and react with our amygdala, but we also have a conscious brain that can override those reactions. And we too live in a world very different from the one we evolved in. Our reflexive defenses might be optimized for the risks endemic to living in small family groups in the East African highlands in 100,000 BC, not 2009 New York City. But we can go beyond fear, and actually think sensibly about security.
Far too often, we don't. We tend to be poor judges of risk. We overact to rare risks, we ignore long-term risks, we magnify risks that are also morally offensive. We get risks wrong -- threats, probabilities, and costs -- all the time. When we're afraid, really afraid, we'll do almost anything to make that fear go away. Both politicians and marketers have learned to push that fear button to get us to do what they want.
One night last month, I was awoken from my hotel-room sleep by a loud, piercing alarm. There was no way I could ignore it, but I weighed the risks and did what any reasonable person would do under the circumstances: I stayed in bed and waited for the alarm to be turned off. No point getting dressed, walking down ten flights of stairs, and going outside into the cold for what invariably would be a false alarm -- serious hotel fires are very rare. Unlike the bird in an aviary, I knew better.
You can disagree with my risk calculus, and I'm sure many hotel guests walked downstairs and outside to the designated assembly point. But it's important to recognize that the ability to have this sort of discussion is uniquely human. And we need to have the discussion repeatedly, whether the topic is the installation of a home burglar alarm, the latest TSA security measures, or the potential military invasion of another country. These things aren't part of our evolutionary history; we have no natural sense of how to respond to them. Our fears are often calibrated wrong, and reason is the only way we can override them.
This essay first appeared on DarkReading.com.
Mossad Hacked Syrian Official's Computer [Schneier on Security]
It was unattended in a hotel room at the time:
Israel's Mossad espionage agency used Trojan Horse programs to gather intelligence about a nuclear facility in Syria the Israel Defense Forces destroyed in 2007, the German magazine Der Spiegel reported Monday.According to the magazine, Mossad agents in London planted the malware on the computer of a Syrian official who was staying in the British capital; he was at a hotel in the upscale neighborhood of Kensington at the time.
The program copied the details of Syria's illicit nuclear program and sent them directly to the Mossad agents' computers, the report said.
Remember the evil maid attack: if an attacker gets hold of your computer temporarily, he can bypass your encryption software.
The Problems with Unscientific Security [Schneier on Security]
From the Open Access Journal of Forensic Psychology, by a whole list of authors: "A Call for Evidence-Based Security Tools":
Abstract: Since the 2001 attacks on the twin towers, policies on security have changed drastically, bringing about an increased need for tools that allow for the detection of deception. Many of the solutions offered today, however, lack scientific underpinning.We recommend two important changes to improve the (cost) effectiveness of security policy. To begin with, the emphasis of deception research should shift from technological to behavioural sciences. Secondly, the burden of proof should lie with the manufacturers of the security tools. Governments should not rely on ecurity tools that have not passed scientific scrutiny, and should only employ those methods that have been proven effective. After all, the use of tools that do not work will only get us further from the truth.
One excerpt:
In absence of systematic research, users will base their evaluation on data generated by field use. Because people tend to follow heuristics rather than the rules of probability theory, perceived effectiveness can substantially differ from true effectiveness (Tversky & Kahneman, 1973). For example, one well-known problem associated with field studies is that of selective feedback. Investigative authorities are unlikely to receive feedback from liars who are erroneously considered truthful. They will occasionally receive feedback when correctly detecting deception, for example through confessions (Patrick & Iacono, 1991; Vrij, 2008). The perceived effectiveness that follows from this can be further reinforced through confirmation bias: Evidence confirming one's preconception is weighted more heavily than evidence contradicting it (Lord, Ross, & Lepper, 1979). As a result, even techniques that perform at chance level may be perceived as highly effective (Iacono, 1991). This unwarranted confidence can have profound effects on citizens' safety and civil liberty: Criminals may escape detection while innocents may be falsely accused. The Innocence Project (Unvalidated or improper science, no date) demonstrates that unvalidated or improper forensic science can indeed lead to wrongful convictions (see also Saks & Koehler, 2005).
Article on the paper.
Why Agile? Why Now? [Matt Heusser's Blog]
I’ve noticed something really odd in the past three months – I’ve started to write articles about Agile Software Testing.
Oh, this is no big surprise, I mean, I work at a shop that releases working software to production every two weeks, uses pair-programming, test-driven development, plans releases using stories and story-points. Typical Agile stuff.
While I have always been interested in lightweight methods, fast feedback, and personal excellence, a quick check of my old published articles finds all of three with “Agile” in the title – one was for a magazine issue with an Agile theme, a second was an interview, and the third, “Peril and Pitfalls of Agile Adoption” was not entirely complementary to the subject.
At the same time, there has been an explosion of Agile Books, Agile Consultants, Conferences and Advice. This morning on twitter, my writing partner, Chris McMahon claimed the Agile Advice ‘market’ was ’saturated.’
It’s pretty easy to argue that if you are a thinker looking to not become part of the herd – that Agile in 2009 is probably not the place to be.
And yet, all of a sudden, here I am talking about Agile Testing. In the two months I’ve published:
Using Automation to speed up Agile Testing
Test-Driven Development Face Off: Part I
Test-Driven Development Face Off: Part II
And I’d like to keep going. Why the sudden energy? Why am I suddenly interested in the ‘Brand’ Called Agile, when I typically suspicious of labels in general?
Here’s what I think is going on. The Agile Manifesto is getting on ten years old. Since the manifesto was developed, we’ve had at least two generations of dogmatic thinking – the XP years, then the Scrum years. We’ve had a fair amount of chest thumping and experts talking about the “right” way to do it and the “wrong way.” We’ve seen practices (someone) doesn’t like labelled #notagile, followed by someone else pointing out that it’s not the practices, it’s the principles, followed by … you get it.
And, quietly, over in the corner, some people were actually shipping working software to customers, and continually getting better and better and it – and – in some cases – developing the skills to talk about it.
After a decade of Agile development, we’ve got anecdotes, we’ve got stories, we’ve got experience. And I can start to reply to the chest-pounding and dogma by saying “really? Let me tell you about my experience. What was yours?”
In other words, I believe the Agile movement has a chance to move beyond dogma and into actually talking about systems thinking and tradeoffs.
We have a chance to have an entirely different kind of discussion around Agile Software Development and, about that, I am excited.
… but what do you think? Has the ‘Agile Software’ movement hit a brick wall or just gotten started? Do we have opportunities to move it forward?
The Inner Ring [Matt Heusser's Blog]
Note: What follows is not original material; it is the transcript of a memorial lecture that C.S. Lewis gave at King’s college shortly after the end of the second world war. I first saw the document at Geocities, one of the original web hosting services, several years ago, and have referenced it many times. On October 26, 2009, Yahoo canceled Geocities and removed references to it from the Google Cache. To ensure “The Inner Ring” does not fade away, I am posting a copy of the “The Inner Ring” here. I hope you find it as valuable as I have.
The Inner Ring by C.S. Lewis
May I read you a few lines from Tolstoy’s War and Peace?
When Boris entered the room, Prince Andrey was listening to an old general, wearing his decorations, who was reporting something to Prince Andrey, with an expression of soldierly servility on his purple face. “Alright. Please wait!” he said to the general, speaking in Russian with the French accent, which he used when he spoke with contempt. The moment he noticed Boris he stopped listening to the general who trotted imploringly after him and begged to be heard, while Prince Andrey turned to Boris with a cheerful smile and a nod of the head. Boris now clearly understood-what he had already guessed-that side by side with the system of discipline and subordination which were laid down in the Army Regulations, there existed a different and a more real system-the system which compelled a tightly laced general with a purple face to wait respectfully for his turn while a mere captain like Prince Andrey chatted with a mere second lieutenant like Boris, Boris decided at once that he would be guided not by the official system but by this other unwritten system.
When you invite a middle-aged moralist to address you, I suppose I must conclude, however unlikely the conclusion seems, that you have a taste for middle-aged moralizing. I shall do my best to gratify it. I shall in fact give you advice about the world in which you are going to live. I do not mean by this that I am going to attempt to talk on what are called current affairs. You probably know quite as much about them as I do. I am not going to tell you- except in a form so general that you will hardly recognize it-what part you ought to play in post-war reconstruction. It is not, in fact, very likely that any of you will be able, in the next ten years, to make any direct contribution to the peace or prosperity of Europe. You will be busy finding jobs, getting married, acquiring facts. I am going to do something more old-fashioned than you perhaps expected. I am going to give advice. I am going to issue warnings. Advice and warnings about things which are so perennial that no one calls them “current affairs.”
And of course everyone knows what a middle-aged moralist of my type warns his juniors against. He warns them against the World, the Flesh, and the Devil. But one of this trio will be enough to deal with today. The Devil, I shall leave strictly alone. The association between him and me in the public mind has already gone quite as deep as I wish: in some quarters it has already reached the level of confusion, if not of identification. I begin to realize the truth of the old proverb that he who sups with that formidable host needs a long spoon. As for the Flesh, you must be very abnormal young people if you do not know quite as much about it as I do. But on the World I think I have something to say.
In the passage I have just read from Tolstoy, the young second lieutenant Boris Dubretskoi discovers that there exist in the army two different systems or hierarchies. The one is printed in some little red book and anyone can easily read it up. It also remains constant. A general is always superior to a colonel and a colonel to a captain. The other is not printed anywhere. Nor is it even a formally organized secret society with officers and rules which you would be told after you had been admitted. You are never formally and explicitly admitted by anyone. You discover gradually, in almost indefinable ways, that it exists and that you are outside it; and then later, perhaps, that you are inside it. There are what correspond to passwords, but they too are spontaneous and informal. A particular slang, the use of particular nicknames, an allusive manner of conversation, are the marks. But it is not constant. It is not easy, even at a given moment, to say who is inside and who is outside. Some people are obviously in and some are obviously out, but there are always several on the border-line. And if you come back to the same Divisional Headquarters, or Brigade Headquarters, or the same regiment or even the same company, after six weeks’ absence, you may find this second hierarchy quite altered. There are no formal admissions or expulsions. People think they are in it after they have in fact been pushed out of it, or before they have been allowed in: this provides great amusement for those who are really inside. It has no fixed name. The only certain rule is that the insiders and outsiders call it by different names. From inside it may be designated, in simple cases, by mere enumeration: it may be called “You and Tony and me.” When it is very secure and comparatively stable in membership it calls itself “we.” When it has to be suddenly expanded to meet a particular emergency it calls itself “All the sensible people at this place.” From outside, if you have despaired of getting into it, you call it “That gang” or “They” or “So-and-so and his set” or “the Caucus” or “the Inner Ring.” If you are a candidate for admission you probably don’t call it anything. To discuss it with the other outsiders would make you feel outside yourself. And to mention it in talking to the man who is inside, and who may help you if this present conversation goes well, would be madness.
Badly as I may have described it, I hope you will all have recognized the thing I am describing. Not, of course, that you have been in the Russian Army or perhaps in any army. But you have met the phenomenon of an Inner Ring. You discovered one in your house at school before the end of the first term. And when you had climbed up to somewhere near it by the end of your second year, perhaps you discovered that within the Ring there was a Ring yet more inner, which in its turn was the fringe of the great school Ring to which the house Rings were only satellites. It is even possible that the School Ring was almost in touch with a Masters’ Ring. You were beginning, in fact, to pierce through the skins of the onion. And here, too, at your university-shall I be wrong in assuming that at this very moment, invisible to me, there are several rings-independent systems or concentric rings-present in this room? And I can assure you that in whatever hospital, inn of court, diocese, school, business, or college you arrive after going down, you will find the Rings-what Tolstoy calls the second or unwritten systems.
All this is rather obvious. I wonder whether you will say the same of my next step, which is this. I believe that in all men’s lives at certain periods, and in many men’s lives at all periods between infancy and extreme old age, one of the most dominant elements is the desire to be inside the local Ring and the terror of being left outside. This desire, in one of its forms, has indeed had ample justice done to it in literature. I mean, in the form of snobbery. Victorian fiction is full of characters who are hag-ridden by the desire to get inside that particular Ring which is, or was, called Society. But it must be clearly understood that “Society,” in that sense of the word, is merely one of a hundred Rings and snobbery therefore only one form of the longing to be inside. People who believe themselves to be free, and indeed are free, from snobbery, and who read satires on snobbery with tranquil superiority, may be devoured by the desire in another form. It may be the very intensity of their desire to enter some quite different Ring which renders them immune from the allurements of high life. An invitation from a duchess would be very cold comfort to a man smarting under the sense of exclusion from some artistic or communist coterie. Poor man-it is not large, lighted rooms, or champagne, or even scandals about peers and Cabinet Ministers that he wants: it is the sacred little attic or studio, the heads bent together, the fog of tobacco smoke, and the delicious knowledge that we-we four or five all huddled beside this stove-are the people who know. Often the desire conceals itself so well that we hardly recognize the pleasures of fruition. Men tell not only their wives but themselves that it is a hardship to stay late at the office or the school on some bit of important extra work which they have been let in for because they and So-and-so and the two others are the only people left in the place who really know how things are run. But it is not quite true. It is a terrible bore, of course, when old Fatty Smithson draws you aside and whispers “Look here, we’ve got to get you in on this examination somehow” or “Charles and I saw at once that you’ve got to be on this committee.” A terrible bore… ah, but how much more terrible if you were left out! It is tiring and unhealthy to lose your Saturday afternoons: but to have them free because you don’t matter, that is much worse.
Freud would say, no doubt, that the whole thing is a subterfuge of the sexual impulse. I wonder whether the shoe is not sometimes on the Other foot, I wonder whether, in ages of promiscuity, many a virginity has not been lost less in obedience to Venus than in obedience to the lure of the caucus. For of course, when promiscuity is the fashion, the chaste are outsiders. They are ignorant of something that other people know. They are uninitiated. And as for lighter matters, the number who first smoked or first got drunk for a similar reason is probably very large.
I must now make a distinction. I am not going to say that the existence of Inner Rings is an evil. It is certainly unavoidable. There must be confidential discussions: and it is not only not a bad thing, it is (in itself) a good thing, that personal friendship should grow up between those who work together. And it is perhaps impossible that the official hierarchy of any organization should quite coincide with its actual workings. If the wisest and most energetic people invariably held the highest posts, it might coincide; since they often do not, there must be people in high positions who are really deadweights and people in lower positions who are more important than their rank and seniority would lead you to suppose. In that way the second, unwritten system is bound to grow up. It is necessary; and perhaps it is not a necessary evil. But the desire which draws us into Inner Rings is another matter. A thing may be morally neutral and yet the desire for that thing may be dangerous. As Byron has said:
Sweet is a legacy, and passing sweet
The unexpected death of some old lady.
The painless death of a pious relative at an advanced age is not an evil. But an earnest desire for her death on the part of her heirs is not reckoned a proper feeling, and the law frowns on even the gentlest attempt to expedite her departure. Let Inner Rings be an unavoidable and even an innocent feature of life, though certainly not a beautiful one: but what of our longing to enter them, our anguish when we are excluded, and the kind of pleasure we feel when we get in?
I have no right to make assumptions about the degree to which any of you may already be compromised. I must not assume that you have ever first neglected, and finally shaken off, friends whom you really loved and who might have lasted you a lifetime, in order to court the friendship of those who appeared to you more important, more esoteric. I must not ask whether you have ever derived actual pleasure from the loneliness and humiliation of the outsiders after you yourself were in: whether you have talked to fellow members of the Ring in the presence of outsiders simply in order that the outsiders might envy; whether the means whereby, in your days of probation, you propitiated the Inner Ring, were always wholly admirable. I will ask only one question-and it is, of course, a rhetorical question which expects no answer. In the whole of your life as you now remember it, has the desire to be on the right side of that invisible line ever prompted you to any act or word on which, in the cold small hours of a wakeful night, you can look back with satisfaction? If so, your case is more fortunate than most.
But I said I was going to give advice, and advice should deal with the future, not the past. I have hinted at the past only to awake you to what I believe to be the real nature of human life. I don’t believe that the economic motive and the erotic motive account for everything that goes on in what we moralists call the World. Even if you add Ambition I think the picture is still incomplete. The lust for the esoteric, the longing to be inside, take many forms which are not easily recognizable as Ambition. We hope, no doubt, for tangible profits from every Inner Ring we penetrate: power, money, liberty to break rules, avoidance of routine duties, evasion of discipline. But all these would not satisfy us if we did not get in addition the delicious sense of secret intimacy. It is no doubt a great convenience to know that we need fear no official reprimands from our official senior because he is old Percy, a fellow-member of our ring. But we don’t value the intimacy only for the sake of convenience; quite equally we value the convenience as a proof of the intimacy.
My main purpose in this address is simply to convince you that this desire is one of the great permanent mainsprings of human action. It is one of the factors which go to make up the world as we know it-this whole pell-mell of struggle, competition, confusion, graft, disappointment, and advertisement, and if it is one of the permanent mainsprings then you may be quite sure of this. Unless you take measures to prevent it, this desire is going to be one of the chief motives of your life, from the first day on which you enter your profession until the day when you are too old to care. That will be the natural thing-the life that will come to you of its own accord. Any other kind of life, if you lead it, will be the result of conscious and continuous effort. If you do nothing about it, if you drift with the stream, you will in fact be an “inner ringer.” I don’t say you’ll be a successful one; that’s as may be. But whether by pining and moping outside Rings that you can never enter, or by passing triumphantly further and further in-one way or the other you will be that kind of man. I have already made it fairly clear that I think it better for you not to be that kind of man.
But you may have an open mind on the question. I will therefore suggest two reasons for thinking as I do.
It would be polite and charitable, and in view of your age reasonable too, to suppose that none of you is yet a scoundrel. On the other hand, by the mere law of averages (I am saying nothing against free will) it is almost certain that at least two or three of you before you die will have become something very like scoundrels. There must be in this room the makings of at least that number of unscrupulous, treacherous, ruthless egotists. The choice is still before you: and I hope you will not take my hard words about your possible future characters as a token of disrespect to your present characters. And the prophecy I make is this. To nine out of ten of you the choice which could lead to scoundrelism will come, when it does come, in no very dramatic colors. Obviously bad men, obviously threatening or bribing, will almost certainly not appear. Over a drink or a cup of coffee, disguised as a triviality and sandwiched between two jokes, from the lips of a man, or woman, whom you have recently been getting to know rather better and whom you hope to know better still-just at the moment when you are most anxious not to appear crude, or naif, or a prig-the hint will come. It will be the hint of something which is not quite in accordance with the technical rules of fair play: something which the public, the ignorant, romantic public, would never understand: something which even the outsiders in your own profession are apt to make a fuss about: but something, says your new friend, which “we”-and at the word “we” you try not to blush for mere pleasure-something “we always do.” And you will be drawn in, if you are drawn in, not by desire for gain or ease, but simply because at that moment, when the cup was so near your lips, you cannot bear to be thrust back again into the cold outer world. It would be so terrible to see the other man’s face-that genial, confidential, delightfully sophisticated face-turn suddenly cold and contemptuous, to know that you had been tried for the Inner Ring and rejected. And then, if you are drawn in, next week it will be something a little further from the rules, and next year something further still, but all in the jolliest, friendliest spirit. It may end in a crash, a scandal, and penal servitude: it may end in millions, a peerage and giving the prizes at your old school. But you will be a scoundrel.
That is my first reason. Of all the passions the passion for the Inner Ring is most skilful in making a man who is not yet a very bad man do very bad things. My second reason is this. The torture allotted to the Danaids in the classical underworld, that of attempting to fill sieves with water, is the symbol not of one vice but of all vices. It is the very mark of a perverse desire that it seeks what is not to be had. The desire to be inside the invisible line illustrates this rule. As long as you are governed by that desire you will never get what you want. You are trying to peel an onion: if you succeed there will be nothing left. Until you conquer the fear of being an outsider, an outsider you will remain.
This is surely very clear when you come to think of it. If you want to be made free of a certain circle for some wholesome reason-if, say, you want to join a musical society because you really like music-then there is a possibility of satisfaction. You may find yourself playing in a quartet and you may enjoy it. But if all you want is to be in the know, your pleasure will be short-lived. The circle cannot have from within the charm it had from outside. By the very act of admitting you it has lost its magic. Once the first novelty is worn off the members of this circle will be no more interesting than your old friends. Why should they be? You were not looking for virtue or kindness or loyalty or humor or learning or wit or any of the things that can be really enjoyed. You merely wanted to be “in.” And that is a pleasure that cannot last. As soon as your new associates have been staled to you by custom, you will be looking for another Ring. The rainbow’s end will still be ahead of you. The old Ring will now be only the drab background for your endeavor to enter the new one.
And you will always find them hard to enter, for a reason you very well know. You yourself once you are in, want to make it hard for the next entrant, just as those who are already in made it hard for you. Naturally. In any wholesome group of people which holds together for a good purpose, the exclusions are in a sense accidental. Three or four people who are together for the sake of some piece of work exclude others because there is work only for so many or because the others can’t in fact do it. Your little musical group limits its numbers because the rooms they meet in are only so big. But your genuine Inner Ring exists for exclusion. There’d be no fun if there were no outsiders. The invisible line would have no meaning unless most people were on the wrong side of it. Exclusion is no accident: it is the essence.
The quest of the Inner Ring will break your hearts unless you break it. But if you break it, a surprising result will follow. If in your working hours you make the work your end, you will presently find yourself all unawares inside the only circle in your profession that really matters. You will be one of the sound craftsmen, and other sound craftsmen will know it. This group of craftsmen will by no means coincide with the Inner Ring or the Important People or the People in the Know. It will not shape that professional policy or work up that professional influence which fights for the profession as a whole against the public: nor will it lead to those periodic scandals and crises which the Inner Ring produces. But it will do those things which that profession exists to do and will in the long run be responsible for all the respect which that profession in fact enjoys and which the speeches and advertisements cannot maintain. And if in your spare time you consort simply with the people you like, you will again find that you have come unawares to a real inside: that you are indeed snug and safe at the center of something which, seen from without, would look exactly like an Inner Ring. But the difference is that its secrecy is accidental, and its exclusiveness a by-product, and no one was led thither by the lure of the esoteric: for it is only four or five people who like one another meeting to do things that they like. This is friendship. Aristotle placed it among the virtues. It causes perhaps half of all the happiness in the world, and no Inner Ring can ever have it.
We are told in Scripture that those who ask get. That is true, in senses I can’t now explore. But in another sense there is much truth in the schoolboy’s principle “them as asks shan’t have.” To a young person, just entering on adult life, the world seems full of Insides,” full of delightful intimacies and confidentialities, and he desires to enter them. But if he follows that desire he will reach no “inside” that is worth reaching. The true road lies in quite another direction. It is like the house in Alice Through the Looking Glass.
Tuesday, 03 November
14:14
Happy Moments in Testing [Testy Testy]
My ten year anniversary is coming up with my company. That doesn't even count time time I was a contractor. It's been nearly 10 years since I stepped into the doors of Adobe on my first day with all of my hilarious assumptions and misconceptions about what working for a software company was all about. I still love my company and feel like I have room to grow as a tester. The happy moments are essential to avoiding burnout I think, and when times get a bit rough it helps to reflect on some of the things that have resonated with me about testing over time.
Moments I like to think back upon and celebrate the most are the shipping of InDesign 2.0. This was the release where we finally took over the desktop publishing market from Quark. The team worked so hard on the quality of this product that I am very proud that we managed to pull off this release. What was special about it is that it truly was hard work, the best work we could possibly do, and ultimately innovation and quality of the product pushed us ahead. It was the integration of native Photoshop files along with better text composition that won that race, and I was a part of it. I really felt that team was strong, and back then success was far from assured. Adobe reached one billion dollars sales shortly after and it was announced and the company celebrated.
As part of that release I had the chance to go to New York city for the first time in my life and help out with the Uncork New York courses. Helping users learn the product and troubleshooting issues. New York pizza really IS that good. I went up to the Empire State Building and saw Ground Zero and this was still 2001, so it was pretty fresh still.
More recently, I've never been more excited or proud than when we shipped Adobe CS3. CS1 and 2 were very hard work, but to go from 4 to 16 products and integrate with Macromedia technology so quickly? That was an insane time, and once again the reason it was a great testing time for me is that I loved the result. The suites are THAT good. And I'm proud to say we reported the top customer issues (all of them) before we shipped. I say that not to gloat, but I'm proud of the job that we did testing that collection of products and the decisions that were made. Now that I think about it, as a test lead, this is the single best moment in my career to date. I felt like the people I'd mentored really flourished and each bit of trust given to them was returned in full. The skills of each team member grew naturally and the leadership of everyone on the team went up a notch. As a lead there is no better feeling than seeing the entire team flourishing.
So, what about now? Now I am getting great job satisfaction when I can solve a new problem, from speaking, from writing, and from finding new ideas to bring back to the testers I work with. When I take a risk, regardless of how it turns out, I'm almost always glad that I did.
There are many great things about testing, even if killing a server, corrupting a document, and coming out of your office holding up the letter V or doing something that looks like a bowling celebration isn't appreciated, take a second and be happy. If your testing job isn't satisfying, I may suggest that there are many things that are optional. Don't make those the most satisfying parts.
There are days when I walk into work and have a whole lot of different things that need doing, none of them short. A typical list would look like:
- first the stuff that's blocking other people
- then the stuff I'm going to forget if I don't do
- then the stuff that others are waiting for but not blocked by
- then everything else.
Dissent and BC's Media [Tyee - Home]
Olympic protesters may be wrong but civil liberties are always right.
Those TV Ads Where TV Companies Bash TV Companies [Tyee - Home]
What's that about? Welcome to the 'fee for carriage' fight, and how to solve it.
Detecting Terrorists by Smelling Fear [Schneier on Security]
The technology relies on recognising a pheromone - or scent signal - produced in sweat when a person is scared.Researchers hope the ''fear detector'' will make it possible to identify individuals at check points who are up to no good.
Terrorists with murder in mind, drug smugglers, or criminals on the run are likely to be very fearful of being discovered.
Seems like yet another technology that will be swamped with false positives.
And is there any justification to the hypothesis that terrorists will be more afraid than anyone else? And do we know why people tend to feel fear? Is it because they're up to no good, or because of more benign reasons -- like they're scared of something? This link from emotion to intent is very tenuous.
Zero-Tolerance Policies [Schneier on Security]
Recent stories have documented the ridiculous effects of zero-tolerance weapons policies in a Delaware school district: a first-grader expelled for taking a camping utensil to school, a 13-year-old expelled after another student dropped a pocketknife in his lap, and a seventh-grader expelled for cutting paper with a utility knife for a class project. Where's the common sense? the editorials cry.
These so-called zero-tolerance policies are actually zero-discretion policies. They're policies that must be followed, no situational discretion allowed. We encounter them whenever we go through airport security: no liquids, gels or aerosols. Some workplaces have them for sexual harassment incidents; in some sports a banned substance found in a urine sample means suspension, even if it's for a real medical condition. Judges have zero discretion when faced with mandatory sentencing laws: three strikes for drug offences and you go to jail, mandatory sentencing for statutory rape (underage sex), etc. A national restaurant chain won't serve hamburgers rare, even if you offer to sign a waiver. Whenever you hear "that's the rule, and I can't do anything about it" -- and they're not lying to get rid of you -- you're butting against a zero discretion policy.
These policies enrage us because they are blind to circumstance. Editorial after editorial denounced the suspensions of elementary school children for offenses that anyone with any common sense would agree were accidental and harmless. The Internet is filled with essays demonstrating how the TSA's rules are nonsensical and sometimes don't even improve security. I've written some of them. What we want is for those involved in the situations to have discretion.
However, problems with discretion were the reason behind these mandatory policies in the first place. Discretion is often applied inconsistently. One school principal might deal with knives in the classroom one way, and another principal another way. Your drug sentence could depend considerably on how sympathetic your judge is, or on whether she's having a bad day.
Even worse, discretion can lead to discrimination. Schools had weapons bans before zero-tolerance policies, but teachers and administrators enforced the rules disproportionally against African-American students. Criminal sentences varied by race, too. The benefit of zero-discretion rules and laws is that they ensure that everyone is treated equally.
Zero-discretion rules also protect against lawsuits. If the rules are applied consistently, no parent, air traveler or defendant can claim he was unfairly discriminated against.
So that's the choice. Either we want the rules enforced fairly across the board, which means limiting the discretion of the enforcers at the scene at the time, or we want a more nuanced response to whatever the situation is, which means we give those involved in the situation more discretion.
Of course, there's more to it than that. The problem with the zero-tolerance weapons rules isn't that they're rigid, it's that they're poorly written.
What constitutes a weapon? Is it any knife, no matter how small? Should the penalties be the same for a first grader and a high school student? Does intent matter? When an aspirin carried for menstrual cramps becomes "drug possession," you know there's a badly written rule in effect.
It's the same with airport security and criminal sentencing. Broad and simple rules may be simpler to follow -- and require less thinking on the part of those enforcing them -- but they're almost always far less nuanced than our complex society requires. Unfortunately, the more complex the rules are, the more they're open to interpretation and the more discretion the interpreters have.
The solution is to combine the two, rules and discretion, with procedures to make sure they're not abused. Provide rules, but don't make them so rigid that there's no room for interpretation. Give the people in the situation -- the teachers, the airport security agents, the policemen, the judges -- discretion to apply the rules to the situation. But -- and this is the important part -- allow people to appeal the results if they feel they were treated unfairly. And regularly audit the results to ensure there is no discrimination or favoritism. It's the combination of the four that work: rules plus discretion plus appeal plus audit.
All systems need some form of redress, whether it be open and public like a courtroom or closed and secret like the TSA. Giving discretion to those at the scene just makes for a more efficient appeals process, since the first level of appeal can be handled on the spot.
Zachary, the Delaware first grader suspended for bringing a combination fork, spoon and knife camping utensil to eat his lunch with, had his punishment unanimously overturned by the school board. This was the right decision; but what about all the other students whose parents weren't as forceful or media-savvy enough to turn their child's plight into a national story? Common sense in applying rules is important, but so is equal access to that common sense.
This essay originally appeared on the Minnesota Public Radio website.
A Quantitatively Managed Process [Matt Heusser's Blog]
There’s a value system out there that says that in order to prove any argument, you need data – and not just any data, but hard numbers. (My father in law, David Ellinghausen, a quality engineer at Ford Motor Company for twenty years, once told me “Without data, your just another guy with an opinion.”)
But let me tell you a story …
I work at a Software As A Service (SaaS) company. We have a website; customers pay us money for a login to the website, then pay for access on a monthly basis, typically on an annual contract. If we provide good service, people renew, and the revenue graph is a big, up-sloping curve. Why, to make money, we don’t have to do anything but keep our customers. Yes – it turns out, as the quality of web apps continues to improve, that’s a bit of an understatement, but I hope you see what I mean.
Now, imagine it’s the beginning of the first quarter of a the new year. We engaged a outside form to perform some research for us – to figure out our customers budgets, their social media strategy, to determine our budget and sales quotas. Picture these two different scenarios:
Scenario 1: The account rep bursts in the room, his face flushed, his tie loosened, with some papers in his left hand. He gulps, looks at the floor, slowly looks up, turns to the CEO and says “We just finished running the numbers. 20% of our customers are not planning to renew next year.”
Scenario 2: The account rep walks into the room, smiling. He shakes the hand of the CEO, gives a high-five to the VP of ops and VP of sales, puts his briefcase on the desk, unbuckles it, and pulls out a glossly, one-page overview, complete with pie chart. “Business is up.” he says “80% of our are customers are planning to renew this year.”
How would you expect the executives to behave in those scenarios? Very differently, right?
… wait a minute …
Those two account executives just said the exact same thing.
As well all know (my friend Michael Bolton pointed it out again last week at STPCon), even when we have them, we do not make decisions based on numbers. We make them based on how we feel about the numbers.
Numbers (quantities) don’t manage processes – people do. Oh yes, there are rare exceptions that are interesting optimization problems. For example, how much inventory to keep on hand for the Christmas rush – too much and you’ll waste money, too little and the shelves will sit empty. But ask a computer to recognize that a shopping mall was just put in accross the street, thus foot traffic will improve … it won’t do it.
For that, you need a human as the master, with the number as the servant, as it should be.
Monday, 02 November
23:58
The Rest of the Product [Abakas]
We have a test plan, and it's great. It covers all the features, and all the workflows of the application. We've got stories, we've accepted them. We've written some automated tests.
- What's production going to look like? How many machines? What configuration?
- How are you going to get the software into production?
- How about the config info? How're you going to go from dev to test to prod? (hope it's not hard coded into the war file or rpm somewhere!)
- Okay, once you got it into production, how're you going to start it?
- Come to think of it, how're you going to stop it?
- One day you're going to have to maintain this thing. Got a plan for that? Is down time okay? Can you do some rolling upgrades or maintenance?
- How are you going to see what's going on? Got logs? Got a way to get logs back to dev for analysis?
- How will you know it's running? Any monitoring? Notifications?
I tend to take notes in meetings. As I was doing that today, it occurred to me that there are different kind of notes that I take:
- All notes all the time. These are extremely extensive notes. It almost gets everything (but not quite - I'm not that fast). I take these kind of notes when I'm not sure what's going to be important. Typically these are Q&A sessions with customers, or a presentation for which I'm totally unprepared. I try not to do this often because it's really hard to listen and take these kind of notes at the same time. If I'm going to have to share notes with people not in the meeting, these are the notes I take.
- Reminder notes. These are much more outline-like notes that I take for most meetings. These are intended to just provide triggers for my memory. These are the notes I take if I need to share them with people who are in the meeting.
- No notes. I do this for a lot of meetings. If I need to be actively participating (or leading) a meeting, I generally don't take notes. I'd rather be fully engaged while I'm there.
We make tool choices constantly, sometimes explicitly and sometimes implicitly. For example:
- I write a bash script to grab some network info off multiple machines. Tool chosen: bash. Didn't even think about it, just did it.
- We're moving parts of our test plan into Jira. Tool chosen: wiki + Jira. This one we discussed for a while, and eventually made our choice based on some cruft with the wiki. I'm not sure it's going to work, but we're giving it a shot.
- I burned a CD of our latest installer. Tool chosen: Disk Utility on my mac. This one is quick and handy, and I haven't gotten a bad burn off it yet.
- What is the tool good for? Jira, for example, is good for workflows. It's horrible for documentation. A wiki is good for documentation but workflow is simply awful. Some tools are more equipped for long term projects and growth than others. Other tools are a lot lighter and good for quick or small projects.
- How convenient is it? The tool I already have will usually trump the tool I don't have, just because of setup overhead. It's not universally true, but it takes a really great feature - or a seriously large annoyance with what I have - for me to switch.
- How accessible is it? Whatever tool I use needs to be accessible to everyone who needs it. IMing out info is no good for my boss, for example, who doesn't use IM. If he needs to know, then I can't use the IM tool.
After a release goes out the door, we hold a postmortem. It's pretty standard stuff, usually. We talk about what we did well, what really didn't work out, and what we didn't anticipate.
- Motivation. People are still stinging from the things we didn't do so well, and are generally aching to fix them. If it was a good release, getting together to remember it will also provide a good boost to people's egos (rightly so!).
- Recency. Memories are better right after the release, and you'll be able to have a better discussion about why you did things and what specifically didn't work about them. You'll have a much more precise discussion while everyone's memories are fresh.
- Ability to change. If you want to make changes, the sooner you start them, the better chance you have.
- Perspective. That process ya'll tried that seemed painful right after you did it might not be so painful now. Maybe you've learned it better, maybe you've started reaping benefits, maybe it wasn't as bad as it seemed.
- Field Experience. Maybe that release that seemed really shaky has performed like a champion in the field. Perhaps that awesome release ya'll tested extensively has had all sorts of problems. These are things you don't know until it's been out for a while.
A lot of derision has (rightly) been spilled on the idea that development writes code and chucks it over the fence at QA. Fortunately, at least in the places I've worked, this doesn't really happen any more. That development-QA fence is basically gone (hooray!).
- Change how we structure our builds to make them more releasable. This is somewhat analogous to writing more testable code.
- Help deploy. Just like our developers do some testing now, maybe we can help deploy, or create some utilities to help. Rake tasks, deployment scripts, hand installations - how does this stuff get deployed, and can we make it easier or better?
- Get help building and packaging. Just like development sometimes asks a tester how best to approach a TDD problem, engineering can get some advice from operations on how to handle a configuration issue, or a packaging question.
- Pair on problems. When there's a problem in the field, we don't have to look in isolation, or bounce questions back and forth. We can work on it together. With two different views and skills looking at the problem, you're more likely to figure out a problem that has a foot in both worlds.
- In-discipline learning is easier and more fruitful. Devs will feed off what other devs are doing, testers will see test innovation and build on it - all because you're working with people who are thinking about the same things you are.
- Allows dynamic resource allocations. If you need an extra tester on a project, great, we can add a tester.
- Explicit thought leadership. You have a head developer who is explicitly charged with improving architecture and development practices. You have a QA manager who is explicitly responsible for evaluating and refining test practices.
- People are serving multiple masters. They're trying to help their teams and also to conform to their (function-oriented) organization structure. This leads to some conflicts of interest.
- Higher risk of silos. If it's a separate group, then you're more likely to have problems with communication.
- You lack predictability in your projects. This happens in consulting a lot, but it can happen in other places, too. If you can't predict how many devs you'll need, it helps to have a pool of devs to draw from.
- You have unusual requirements on one or more of your groups. If you're doing some really unusual testing, for example, you may need to keep your testers together so you pick up the learning and innovation effect.
- Avoid this if you're attempting to embrace SCRUM or some other cross-functional team ownership mentality. The "multiple master" problem will get you in this case.
- Unity of purpose. The team is all working toward the same goals. There is no secondary or other goal.
- Breakdown of silos. If you can get true team ownership, you start to find developers testing, and testers helping with product management, etc.
- No need for functional management. The role of "QA Manager" goes away here. Instead you have team leads.
- Harder to drive functional change. When you have several teams with a few testers each, it's a lot harder for testers to innovate or learn from each other. The same goes for developers. The groups are simply too small to get that kind of momentum.
- Hard to handle changing needs and moving through software development phases. You run the risk of having idle testers as you start an effort, and idle developers at the end of an effort. This is something that can be overcome, but you have to encourage cross-functional work, and be sure to plan appropriately.
- You're using SCRUM.
- You can have generally stable teams. This implies your projects (or products) are pretty consistent in size and resource needs.
- Avoid this if you have a particularly weak functional area (or more than one). There's a large risk that isolation within a stronger team will make them even weaker.
We work in what I imagine is a fairly typical environment. We code away on HEAD for a while. Once we're feature complete, we branch (so now we have HEAD and RELEASE). Then we fix stuff on HEAD and merge it to release until we hit code complete. We also go ahead with the next features on HEAD, but that's not currently the point.
- Code changes to production code. Bug fixes, new features, etc.
- Code changes to test code. Change the tests, not the code that actually ships.
- Infrastructure changes. Change something underlying about the lab or environment (e.g., update the default fstab that gets installed)
- Is the change caused by a failure or just a cleanup/enhancement/nice to have? The former is more likely to get put in than the latter.
- Is the change going to fix something that causes problems for other tests? (e.g., a hang that stops all later tests in the suite from executing). A bad citizen like that is more likely to get fixed.
- Is the change risky? The same types of analysis apply here as for product code. Avoid big, sweeping, likely-to-break-something changes.
- How many more test runs are we going to have? The closer we get to the end, the longer we can just deal with the problem and not bother to fix it.
All The Other Tests You Did [Abakas]
I've been verifying bugs for the past day or so. It's actually work I really enjoy. The vast majority of the time, it's concrete evidence of the product being better, which is awesome. Plus it's very easy to see the progress I'm making, which appeals to the list maker in me (I love checking things off!).
- install two systems
- create a volume on one system
- configure replication between the two systems
- configure replication on the volume
I don't drive very often - I live and work within the city, and I tend to take the T to work and pretty much everywhere else, too. Consequently, I don't listen to the radio very often. But this weekend I was on a road trip and I had the radio going. I kept hearing the same theme over and over as I flipped around the stations:
- some of them are the same over and over again: how many weeks is one song at the top of the charts, or at position two or five? Same thing with areas of code.
- some of them change each time: eventually a song falls off the list, and eventually we're comfortable enough with an area of our test plan that we move on.
- they sound a little different every time: maybe this week it's the dance mix and next week it's the acoustic version - same underlying thing, just a bit different.
As we're testing our software, we have lots of different kinds of requirements. We have use cases, functional requirements, performance requirements, usability requirements, testability requirements, etc. One of the requirements that we don't usually talk about explicitly is the good citizen requirement.
- support logging in a common format (e.g., NT Event logs, etc)
- use centralized user or machine management (e.g., Active Directory or NIS)
- does automatic log rolling
- can be configured to start on its own after a power outage or other event
- can be disabled or somehow turned off cleanly (to allow for maintenance, etc)
- log excessively (at least, except maybe in debug, which should be used sparingly)
- create excessive traffic on infrastructure servers (DNS, Active Directory, mail, firewall, etc)
- send excessive notifications (e.g., a notification for every user logging in would probably be overkill)
We're a storage company. Lots of people write to us with lots of different programs. These run the gamut from drag-and-drop to homegrown bash script to full HSM solutions, and lots of points in between. Sometimes we'll get asked to "certify a client" application. What's going on here?
- The standards approach. This is where you point to some external standard and say, "we conform to this. Any client that works with this will work with us." By external standard, you should make sure you choose a public standard: NFS v3, or W3C compliance, or whatever's appropriate for you. In this case, you don't actually have to test the client. However, you'd better be darn sure you conform to the standard, or this one may eventually bite you.
- The "we test this" approach. This is where you offer up the version and configuration you test, and you say that has been tested and will work. Any deviance from that configuration or version may work but isn't guaranteed.
- The "certification program" approach. This is where you turn it around on the client application, and offer a certification program. The idea is that they conform to you, rather than the other way around. You offer a set of criteria, test systems (or a lab for people to come test in), possibly scripts and reporting mechanisms, and you let people run your tests. Then you analyze their results, and either put your stamp of approval on or not (think "runs on Vista", etc). If you're large enough and important enough, people will do the compatibility testing for you. This doesn't work so well if you're kind of a tiny nobody in your industry. I've not done this one personally.
- First, resolve compilation errors. If it doesn't compile, not much is going to run.
- Second, resolve bugs that cause machines to leak. If a test causes machines to not clean up after it's done, then they're not available for other later tests. This causes the entire lab to grind to a halt. Generally this is accompanied by cries of "we're out of machines!" and tests just not finishing because there's no machine for them to run on.
- Third, resolve bugs that are likely to hide other bugs. If I have a bug in my test setup, who knows what will happen when I get to actually exercising the thing I thought I was testing!
- Fourth, handle everything else. Once you've gotten through the first three items, then just start fixing bugs according to your preference.
At any point in time, when I sit down at my desk, there are about fifteen things I could do. For example, when I sat down after today's dev leads meeting, I had my choice of:
- verify some bugs targeted for the next release
- verify some bugs on head
- answer a question from a sales engineer
- review a potential client sizing worksheet
- fix one or more of about three small-ish bugs assigned to me ("fetchlogs doesn't fetch when... whoops!")
- read a white paper
- work on an in-progress FAI server I'm configuring
- active fires
- stuff for clients
- stuff for sales
- stuff other people on my team can't do
- other stuff
I just read this article by Joanne Rothman, talking about a "low level buzz" with chats, emails, and talking going on all the time. She basically says that maybe having a chat open all day, or high email traffic with quick responses might work for some people but not for her.
- build failure notifications, and their causes
- discussion of which branch to run in the nightly automated tests
- notification when weekly lunch arrives (the important stuff!)
- code review requests
- heads up about interesting or risky checkins
- pairing requests
- general pleas for help (e.g, "how do I do X in perl?")
Just like there are some magic numbers that can point you to the source of a problem, there are some magic words that also have power. The words you use to describe a problem will color how other engineers (developers, other testers, managers, even you) look at the problem and what they do to track it down.
- Crash. Most people will be looking for a core file and an uncontrolled shutdown in this case.
- Hang. Often you'll get asked how long before you mark something as being hung. Also, this word means that absolutely no progress is being made. If it's progressing at a snail's pace, it's still not a hang; it's just really slow.
- Failed. This one means completed with error. It doesn't mean "still going".
- Wedged. This is imprecise but generally seems to make people think deadlock.
- Data loss. The system could lose data in this scenario (YIKES!). This one raises red flags all over the place.
- Failure. This word means system change that resulted in the loss of availability of a system component. Generally its used for hardware failure (of a disk, node, power supply, network connection, etc). Generally "failure" does is modified by some component name (e.g., power supply failure), and is not used for the more general "software didn't work" case.
- Simultaneous Failure Vs Sequential Failure. In a redundant self-healing system like ours, number of failures (see above) matters, as does whether the second one occurred at the same time as the first or after. Depending on which one happened, a whole different debug path will be invoked.
There are some numbers that I call magic numbers. These are the special numbers that have meaning in the context of your system. These numbers are typically diagnostically important, or triggers to identify problems or potential problems.
- 86400 seconds: As in, "the test timed out after 86400 seconds". This is a day. As in, the darned thing didn't finish up in a whole day. Oops.
- 2^32 or 2^32 -1: If you're on a 32 bit system, you're starting to wrap address space here. Look for really large negative numbers where you're expecting positives, etc.
- 45 seconds: the default connection timeout for network mounts in Windows. If something fails after this long, you're looking at a timeout probably.
- 6 minutes: The RPC timeout for a query to a remote system is 2 minutes, and it does 3 tries. 6 minute waits mean you're hitting this.
- 5 minutes: The frequency of heartbeat checks for certain operations. Failover after 5 min means these never succeeded.
- 25: default max number of simultaneous connections. Can be increased indefinitely, but if you start to see slowdowns or connection timeouts and you have 25 clients in use, you're probably going to want to change it.
- XX: Java heap size. (I don't remember this one off the top of my head, but I know it when I see it)
- Both of us open our various browsers
- Both of us point all those browsers to the developer's machine. Now we can see everything in all browsers pretty quickly.
- We make sure no one's caching CSS or JS (Often this is a configuration setting on the development environment, and probably already set up, but it's worth checking.)
- Developer makes a change.
- We both reload all browsers. Nope. Not there yet.
- Developer makes a change.
- We both reload all browsers. Success on IE7, but we broke Firefox 3. Darn.
- Developer makes a change.
- We both reload...
- ... etc etc etc ...
- Developer makes a change.
- We both reload all browsers. Success!
- Check in.
- Any mention of times in the project. Consider backup windows, data flows that include your system (e.g., stored on primary for 30 days and then on your backup product for 60, then on tape forever - means you have to be able to take in one day of generated data every day because that's what'll come off primary to keep primary at 30 days), the speed of current solution (if any), any number you can get. From this, you can calculate performance requirements. Number of messages passed over an interface divided by hours available for batch feed processing equals required messages per hour for you.
- Find comparables. Most things, particularly performance, are relative. If a customer is opening a file off your system, look at how fast the customer can open a file off another network drive. That's your implied performance requirement. If a customer can open his web-based EMR page in 3 seconds, then your web-based scheduling system probably ought to load pages in about 3 seconds. You can be off from comparables a bit, but you want to be pretty close to keep your customer happy.
- GUI management
- CLI
- Test Infrastructure
- HA
- Replication and snapshots
- ...
Your Customer's Wants [Abakas]
There are many many people in this world who care about what we do as testers and as software engineers, in one way or another. A significant subset of these are people we would consider customers. Our customers are the people that consume our work. And they have wants and needs.
- information about the product's current state
- information about the severity and likelihood of an issues they have/might/hope they won't find
- test coverage information
- risk assessment of our products
- Zero defect releases
- The ability to "make up for engineering" by testing in some quality (darn it!)
- a go/no go release vote
- something to measure how good their testers really are; a "standard" or "guideline" or "best practice" or "certification" or "metric" (the words are different; the desire is the same)
- There's no tester consensus on this one. Some schools of thought say that test is an information providing function only and should not make a decision about the location of the software in the development life cycle (moving through any phase, including release). Other schools of thought embrace the notion of testers as a hub of information and therefore put the release decision squarely in the test corner.
- This isn't about something industry-wide. For every "testers are information providers" argument there's a "testers have a place at the release table" argument, and they're both valid. These are arguments about role and function within an organization, which means they're specific to that organization. What they do at "other company X" is only marginally relevant.
We've been building software for a while, and we've been using stories for a while. You'd think estimating would get easier, but at least for me it really doesn't. There are several techniques we use to create test estimates, and I thought I'd share another one.
- Estimate your current stories. Do this with whatever model you like, just get to the "this will take X" point.
- Go through past stories and group them into "big" "medium" and "small". This is a grouping of test effort here, and it reflects how hard it looked to test. Your gut feel applies here, and you're welcome to include other metrics (e.g., "that team does a ton of refactoring, so their stories are always medium or larger"). Be sure to do this over as long a period of time as possible, so you can flush out any really weird circumstances.
- For each story, determine how long it actually took you to test. Use calendar time here: from the day you started until the day you stopped. If you had it in test multiple times (test it, failed, retest it), count them all up together.
- Do a calendar-to-estimate ratio. Let's say your calendar was 8 days, and your estimate was 8 hours. Congratulations, one of your estimated hours is a real world day in this case. Calculate this into the form 1-hour-to-?? (e.g., 1 hour is 1 day).
- Average the estimates within the buckets. At this point you have a list of calendar-to-estimate ratios. They probably look something like this: 1h:2h, 1h:1day, 1h:3days, 1h:4h. Now we simply average these. Add 'em all up, divide. If you're feeling fancy, throw out the biggest and smallest outliers. The result is a single ratio: 1 hour estimate time is ??? calendar time. This is your real estimate.
- Adjust your estimates. Now, go back to your current story estimates. If, for example, on medium stories, your average ratio is 1 hour to 4 hours, and your current estimate was 2 hours, then your new estimate is 8 hours.
- Create atomic change sets
- Isolate your change sets
Sharing Feedback Systems [Abakas]
Pop quiz:
- No more copying and pasting from emails or one system into another. You've just bought someone hours of time!
- Fewer duplicates. Your tester will confirm things your customer saw rather than double-reporting them, and (hopefully) vice versa.
- More comfort. Your customer knows exactly what he's getting and gets some comfort that you're not just writing and waiting on him, but that you're actually using this and working out the kinks before it goes live.
- All your warts? Totally visible. This is really the big downside here.
- You all have to watch your language a bit. No putting "the customer is always right, even if they're total idiots" in bugs.
- Less comfort. You mean you're not perfect? If the developers didn't catch this, then what are they missing?
23:57
testingjeff [Jeff Fry on Testing]
More and more web applications are taking advantage of asynchronous JavaScript (or AJAX) – and with good reason. It gives tremendous power to a web application, allowing the app to respond fluidly to the user without requiring a full page reload and making it feel more like a native desktop app. At the same time, it makes the application trickier to automate.
When I first started writing browser-based test automation, it was safe to assume that once the page loaded, if an element wasn’t there yet, it wasn’t going to be. With AJAX, that assumption goes out the window. Let’s take an example:
Someone’s registering for our website. The site prompts her for a username, which she enters. Now, in the olden days, she would fill out the rest of the page and then submit. The page would refresh…and often tell her that the username she selected is unavailable. Rinse and repeat until you finally find an available name.
With AJAX, the moment she clicks away from the username field, we can have feedback appear on the page, letting her know whether or not that name has been taken. She doesn’t need to wait for a page refresh before each attempt, the feedback may be nearly instantaneous.
Sounds good, right? Now I come along to automate a test of this page. If I write something like (in pseudocode):
set username = reserved_name
assert response_text says reserved_name is taken.
…much of the time, I’ll get an error saying that the response_text isn’t there on the screen.
How can I tell my script not to progress to the next step until the application is ready? I have a range of options here. At the cringe-inspiring end of the spectrum, I can write:
set username = reserved_name
sleep 10 seconds
assert response_text says reserved_name is taken.
The problem here is that one either sets the sleep too short, and the script errors out when the test server is under a bit of extra load — or one sets it too long, and the scripts begin to spend 90% of their time sleeping.
Somewhat better is to do like so:
set username = reserved_name
wait( up to 20 seconds) for response_text #then
assert text in page says reserved_name is taken.
The issue here is that you need to know exactly what you’re waiting for, may need to list multiple triggers to wait for, and your script risks of aging poorly as these triggers change.
To get better than this likely requires testability features in the application under test. One thing that’s worked well for me is having the application set a flag whenever it starts to execute AJAX, and unset that flag when the AJAX completes. At Freebase.com, we chose to have that flag be an attribute on the body tag, like so:
Whenever AJAX is kicked off, this is set to ajaxStart and whenever the last asynchronous process completes, it sets it to
Now I can say something like:
set username = reserved_name
wait_until_loaded
assert text in page says reserved_name is taken.
Where wait_until_loaded is defined like:
Wait( up to 20 seconds) while body.attribute(ajax) == ‘ajaxStart’
This pushes the burden of knowing when the page is ready for the next step back to the application under test. The downsides of this approach are: (1) it requires code in the application under text to make it work. Sometimes that’s difficult (politically or otherwise). (2) this solution’s only as robust as the code that sets to ajax flag. If the application under test can’t or won’t report this reliably, this solution will only cause you grief.
That said, the upside of this approach is potentially huge: It frees the tester from writing a custom wait trigger after every AJAX step, speeding up test script writing and often making the scripts more robust as well.
Depending on how familiar you are with the testing library you’re using, it may be worth considering putting this check deeper in your code. For example, one might change the click method to always wait_until_loaded before clicking. This has the potential to streamline test scripts that much more.
In fact, the Windmill test automation library has already done this on trunk. As of the next release of Windmill, the default behavior for everything other than asserts is to wait for that element for a set number of seconds — progressing immediately if it’s there right away, but only throwing an error if the timeout is reached. I suspect as web applications become more AJAX-heavy, this will become the standard in test automation libraries.
There isn’t a One Right Answer for any context, but I’ve been very happy with having the application under test set an ajax flag of some sort. If you’re testing an application where the page changes asynchronously, I encourage you to give it a try.
testingjeff [Jeff Fry on Testing]
Software Testing has many joys. Most have to do with solving tricky puzzles, collaborating with brilliant minds, or contributing to the creation of elegant software…but sometimes there are simpler pleasures as well.
Today a colleague sent around a prototype of an application that takes any string and attempts to pluralize it. It handles a lot of tricky cases, e.g. correctly pluralizing “surgeon general” as “surgeons general”. On the other hand, it makes many mistakes, and I suspect will always do so. One example: it currently assumes that any input is singular. When run against one of the data sets it was designed for (a list of Types Freebase.com users have created) I noticed it turned “Most Disliked Things” into “Most Disliked Thingses”. At that point I couldn’t resist watching it turn “Hobbits” into “Hobbitses”…and then coining a new term:
Smeagolize: To alter a word or phrase, making it sound like something the character Smeagol from the Lord of the Rings would say, e.g. “Most Disliked Things” into “Most Disliked Thingses”
This reminded me of one of my favorite examples of a coder solving a problem in an amusing way. This was many years and companies ago. A harried coder was asked to display some (potentially long) URLs in a very small space on a home page. As he worked on it, he discovered that some URLs didn’t fit the space. What to do? A less intrepid coder might have asked for help with the design problem, but he tackled it himself…by writing code to remove all the vowels from any URL over 20 characters. This led us to coin the term:
Disemvowel: To remove the vowels from a URL or other word, e.g. changing
http://www.associationforsoftwaretesting.org/ to
http://www.ssctnfrsftwrtstng.rg/
Needless to say, this particular code didn’t live to see the light of day…but its memory lives on in the term Disemvowel.
Has broken software inspired new words for you as well?
testingjeff [Jeff Fry on Testing]
Here’s a 22 minute video lecture on using controlled experiments to discover what your customers think, and if you work on web software I suspect you’ll find it 22 minutes well spent. Kohavi points out several examples where the results of tests were quite surprising, as well as some interesting suggestions for how to organize them.
Note, this is related testing software in the broad sense. The bugs that split tests find are the (plentiful) bugs in our mental models, not in our implementation.
testingjeff [Jeff Fry on Testing]
If you haven’t been to Stack Overflow yet, it’s an interesting forum for asking technical questions — and sorting through the answers — written by Joel Spolsky and Jeff Atwood.
I noticed a question on Model-Based Testing over there that I had something to say about. I wanted to link to articles by Harry Robinson, Ben Simo and James Bach…but as a new user, I’m allowed to add only one link. What to do? How about using my one link to go to my blog. Here’s the original question.
And here’s my answer, complete with links:
First, a quick note on terms. I tend to use James Bach’s definition of Testing as “Questioning a product in order to evaluate it”. All test rely on /mental/ models of the application under test. The term Model-Based Testing though is typically used to describe programming a model which can be explored via automation. For example, one might specify a number of states that an application can be in, various paths between those states, and certain assertions about what should occur in on the transition between those states. Then one can have scripts execute semi-random permutations of transitions within the state model, logging potentially interesting results.
There are real costs here: building a useful model, creating algorithms for exploring it, logging systems that allow one to weed through for interesting failures, etc. Whether or not the costs are reasonable has a lot to do with *what are the questions you want to answer?* In general, start with “What do I want to know? And how can I best learn about it?” rather than looking for a use for an interesting technique.
All that said, some excellent testers have gotten a lot of mileage out of automated model-based tests. Sometimes we have important questions about the application under test are best explored by automated, high-volume semi-randomized tests. Here’s one very colorful example from Harry Robinson (one of the leading theorists and proponents of model-based testing) where he discovered many interesting bugs in Google driving directions using a model-based test (written with ruby’s Watir library).
Robinson has used MBT successfully at companies including Bell Labs, Microsoft, and Google, and has a number of helpful essays.
Ben Simo (another great testing thinker and writer) has also written quite a bit worth reading on model-based testing.
Finally, a few cautions: To make good use of a strategy, one needs to explore both its strengths and its weaknesses. Toward that end, James Bach has an excellent talk on the limits and challenges of Model-Based Testing. This blog post of Bach’s links to his hour long talk (and associated slides).
I’ll end with a note about what Boris Beizer calls the Pesticide Paradox: “Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffective.” Scripted tests (whether executed by a computer or a person) are particularly vulnerable to the pesticide paradox, tending to find less and less useful information each time the same script is executed. Folks sometimes turn to model-based testing thinking that it gets around the pesticide problem. In some contexts model-based testing may well find a much larger set of bugs than a given set of scripted tests…but one should remember that it is still fundamentally limited by the Pesticide Paradox. Remembering its limits — and starting with questions MBT addresses well — it has the potential to be a very powerful testing strategy.
testingjeff [Jeff Fry on Testing]
On the watir-general email list, George Sanders recently wrote
“It seems that I’ve been encountering more people within my workplace (and, alas, even within my own QA team!) that are not sold on test automation. From what I’ve learned so far, there seems that automation will never cover 100% of what needs to be tested, but this doesn’t negate the need.
Another frustration is that I’ve been tasked to write automation scripts as part of my year-end goals. However, I haven’t been assigned hours in my work week to do them! All of my script development has been after-hours and weekends (notice I’m posting this on a Saturday!).
Has anyone else run into naysayers? How can I convince the decision-makers that this is a worthwhile effort?”
I responded on-list, but want to improve my answer (and change my analogy). This conversation tends to be quite unproductive when it becomes “Is or isn’t test automation useful?” It’s a lot like “Are ladders useful?” Ladders won’t solve any problems on their own. There are plenty of problems that ladders will not help with a bit. Problems that are helped by step-ladders may only be made worse by a 20 foot ladder. Yet there are some problems where the right ladder well used will add tremendous value.
Similarly with testing, I like to look at what are the problems that need to be solved. Then I like to think about each of them, and consider solutions. Is one of your problems basic functionality breaking when code is changed? Unit tests might be a great solution to your problem. Do you suspect intermittent production failures are the result of concurrency issues? Likewise, automation-assisted tests might really help in reproducing the problem in house.
I think discussing the relative costs/benefits of automated browser-based regression testing is a good idea, and getting real experience reports helps a lot. Within this realm in particular — depending on many contextual factors — there may be some problems that’re helped through automated browser-based tests and others where the cost is too high.
Note, for some concrete examples of problems where Watir has been useful for me, see my post about a few of the most compelling cases where I’ve used Watir over the past five years.
Getting time added into the schedule for test automation is a different question, but one that might become easier if you’re able to focus the conversation around solutions to particular problems. (On the other hand, there are a lot of reasons — sensible and not — why test automation might not get onto the schedule.) In any event, focussing the conversation on specific problems and potential solutions for them is likely to increase the odds of a productive conversation.
testingjeff [Jeff Fry on Testing]
A sign of the times, I know a handful of great testers and coders who’ve been laid off in recent months. One I just heard about today is Chris McMahon. I first encountered Chris as a contributor on the Watir, Software-testing, and Agile-testing mailing lists. At the time, I was QA Manager for a company that was having a rough time filling a position for a very technical API tester and test automation expert. I emailed Chris, one thing led to another, and I had the good fortune to work with Chris for a period.
Several jobs later for both of us, he fell prey to economic downsizing today. I have to say I was very impressed (though not really surprised) to see what a stir it has made in the software testing communities I’m a part of. The impact is clearest in all the conversation about and testimonals for Chris on Twitter.
Is this because Chris is a very skilled tester? Absolutely it is…but there are other very skilled testers out there who just aren’t as known. He blogs, he has had articles published in several journals, and he actively contributes to multiple online testing communities. And by contributes I mean he engages in dialog, he offers ideas, he offers help to folks who ask good questions. On the Watir list, he claimed an unofficial spot some time back as the Answerer Of Off Topic Questions. When someone raises something that’s more of a ruby / test design / other library related question, he has frequently had something helpful to contibute, and he’s done so, even as he worked for the last year or so at a company that uses Selenium instead.
Being a part of a testing community has many benefits: Exposure to new ideas, meeting colleagues, a chance to have our ideas tested and improved via feedback, etc. — but this is a place where it’s particularly clear. It may turn out to be a tough time to be looking for a remote testing position, but the way Chris has chosen to live his professional life over many years seems to be reaping major dividends right about now.
testingjeff [Jeff Fry on Testing]
I’ve just started my second round of teaching Black Box Software Testing Foundations for AST members. Besides the obvious benefits of what I consider to be a very well designed and rigorous course, there are the side benefits of meeting interesting folks from all over the world, and the discussions that arise. One student who’s currently between jobs, and has extensive experience testing embedded software but is looking for a job testing web applications asked me for advice.
Looking back on my answer to her, it seems like something others might appreciate as well, so I decided to publish it here as well.
Specific ideas for transitioning from testing one type of software to another:
- Consider finding a relevant open source project to help test. I can’t think of an open source web app right now, but certainly experience testing Firefox or Watir sounds relevant to me…and even experience testing something further afield like Open Office shows that your experience is applicable beyond embedded software. The bug reports you file here become a publicly visible portfolio that you can link to in your resume.
- Consider reading/learning more about web testing in particular. You might add some of what you’ve read to your cover letter or resume directly, or perhaps it will just inform how you answer questions when you get called for an interview.
- Finally, if you’re having a hard time getting the permanent job you want, consider getting a short-term contract to get web testing onto your resume.
And then there’s the general advice:
- Make your resume & cover letter shine. When I’m hiring, cover letters that convey personality, communication skill, and intelligence are few and far between. I am *much* more likely to call a candidate with holes in her/his resume if their cover letter is strong – which certainly includes enough personalization that I can tell s/he has thought about working at *this* company in particular.
- Are you not getting called for interviews? Ask friends or colleagues who you can trust to be thoughtfully candid to review your resume and cover letters.
- Are you getting interviews but not getting offers? Ask the hiring manager why. Five years ago, when I was transitioning from my first testing job and looking for my second. I got turned down for a gig I was interested in after the second interview. I worked up my courage, called the hiring manager up, and said I wanted to know why I hadn’t gotten the position so that I could learn and improve myself. He was very impressed…and in fact after talking for 10 minutes on the phone told me he’d changed his mind and made me an offer. While I can’t gaurantee that that’ll happen for you, I think there’s at least a decent chance you’ll learn something. As a hiring manager, I’ve had a lot of candidates who turned me off because of something they wrote or said, or who I turned down because of a skill they appeared to lack. I don’t believe it’s my business to point this sort of thing out unsolicited, but if someone takes the initiative to ask I will often be happy to offer a friendly tip or two for the next place they apply.
testingjeff [Jeff Fry on Testing]
David Christiansen, the editor of the AST Update, has put out a nice new issue…including an essay of mine on working with Inattentional Blindness. It’s available in print and as a PDF, and there’s a lot of good in it. Check it out!
testingjeff [Jeff Fry on Testing]
I’ve been home from the 3rd annual Conference of the Association of Software Testing in Toronto for three weeks now, and am still thinking about all I’ve learned.
This was my 3rd time attending CAST. As always, the keynotes, tutorials and track sessions were excellent…and as always, even better than that was the conferring. You see, CAST knows that it attracts testers with an impressive array of experience, and that making time for them to riff off of each other is very valuable. Toward that end, every single track session and keynote included substantial time devoted to discussion, with an explicit welcome to challenge the presenter or to otherwise test their assertions. Inevitably, the I found the lively conversation spilling over into dinner, or following us to the pub.
While I’ve been testing for eight years now, and have worked on a range of web and client-server applications, all of my work so far is limited to the San Francisco Bay Area. It’s a tremendous pleasure discussing testing with folks from vastly different industries, and from around the world.
One story of many: Over dinner I mentioned a problem that I thought was best solved by starting a conversation with the programmer. Scott Barber responded with something like “Providing the tester doesn’t get in trouble for talking to her.” I was surprised, and asked if that’s really something that happens much in 2008…and was informed by folks at the table from more regulated industries or on government projects that that’s not uncommon. Partly this made me very happy to be a tester for in very human culture we have at Freebase, but it also served to reinforce how easy it is to overgeneralize about the field of testing, when I’m really thinking about testing in a particular context or contexts.
Discussing our particular challenges, sharing stories, and questioning testers with very different perspectives really made CAST 2008 a joy for me…and I’m looking forward to 2009 in Colorado Springs!
testingjeff [Jeff Fry on Testing]
This is only loosely related to software testing…being some of the things that keep me healthy while I do it. This started as an email template, which I’ve sent to many friends and colleagues at their request – enough that I decided it was worth posting here:
Keyboard
The Goldtouch Adjustable Keyboard is what I use at work, and like it a lot. I found it to have next to no learning curve. I know others who prefer the Kinesis Contoured Keyboard…which friends say takes up to a week to learn to use, but I know a couple folks who my keyboard did nothing for, and who love these.
My biggest advice for folks though is to try several options out, and (for folks in the SF Bay Area) Office Relief is a great place in San Leandro to do so.
Mouse
Again, very personal, and trying out options is definitely the way to go. What I’ve settled on is a fairly traditional mouse on my left, and an Evoluent VerticalMouse 3 on my right. Some days I use mostly one or the other, most days I switch between them. Note, I was excited to try their left handed version, but it is still at v2 and I really didn’t like the way it fit in my hand.
Taking Breaks
The last thing I’ll say is that the single most helpful ergonomic adjustment I’ve made is software that helps me to pace my breaks. There are good free options for PC (Workrave) or Mac (Anti-RSI). Both are very configurable.
23:53
People who make it so worthwhile. [Testy Testy]
Today was a banner day for me at work in some ways. Marlena Compton came to speak at Adobe and did a fantastic job! Seeing people really consider her new ideas and start to imagine where it could lead was just inspiring. Her work is scratching the surface of something so needed in software testing. It made me so happy to be able to see other people catching on to where this could lead. It isn't just the same old testing ideas rehashed. There is some new innovative thinking going on here and I think the newsmap she showed was fitting for the audience as it is made with Adobe Flash.
Anyhow, the other person who has really changed how I think about collaboration lately is a gentleman (and I do not use the term loosely) by the name of Matt Heusser who collaborated on some writing with me, and I must say it was the most natural collaboration with another tester I've experienced in ages. This guy is just a delight to write with and if you ever get down on human nature or feel that all testers are cynics and grumps, I urge you to go read Matt's blog because you will leave with a different attitude about the state of testing and a better and more balanced one at that.
So, you might think that today's blog is sponsored by the Letter "M" or people beginning with "M", but I file it under testers who I will work hard to deserve their respect now and in the future.
Are you any good at speaking? [Testy Testy]
Both times I've presented, afterward I wonder if I'm any good at it. It is quite hard to separate form and content when it comes to speakers, and since I talk on topics I'm very passionate about, usually I don't have people leave because they are bored. Those who were expecting someone as good as one of the keynote speakers? Well, I think I let some of those people down.
So, I enjoy speaking, but does that make me any good at it? Compared to the keynote speakers, I am not good enough. I get nervous, I talk too fast, tell many stories, and am not as orderly as I intend sometimes. I sometimes say things I wish I hadn't in retrospect because I'm too candid. I'm politically imperfect, that's for sure, but I think I am a speaker with potential. I need more experience, I need more feedback, and I need to see more GREAT technical speakers who I admire and respect.
For my second time speaking in public ever and my first time presenting Reducing Test Case Bloat (dry run I had to cancel due to sudden surgery) I am very proud. I think I did very well. I don't know how good of a speaker I can be yet. However, I'm presenting Reducing Test Case Bloat version 2 at Microsoft on Friday the 13th of November. I'll be adding in more detail on the "how" part because most of the suggestions were already on my list, so I want to convey the techniques this time, not just get people to read the paper. This time I'll ask anyone to mention suggestions during the Q & A portion if they have them and write them down rather than taking time out.
Despite turning on the capcha, I still have spam which is annoying! No idea besides banning and marking as spam how to stop it further. Sorry for anyone who got emailed.
This morning has set itself against me. After spending money to make my clever pun based halloween costume decent for work (you ladies know what I mean) by adding about DOUBLE the fabric (leotard, tights, shorts, longer skirt, etc), the stupid zipper broke UNZIPPING it to put it on before I ever had it on. Quality, what? Ok, then the spammers got to my blog and emailed real people I respect despite my efforts.
Let's hope that there are no more tricks and far more treats in store for me this weekend.
In good news, Marlena Compton is speaking on Monday, revising her PNSQC talk just for my fellow QE! I'm so lucky she is in town. She's really smart and working on visualizing data for her Master's. Also, we met because she's a reluctant twihard who is on a pilgramage to Forks, WA.
PNSQC 2009-Presentation [Testy Testy]
I had a fantastic time at PNSQC to the point I'm just worn out! I could not sit down and behave myself despite having recently had surgery, so it took me a few days, however, I have now prepared the reducing test case bloat technical paper along with the reducing test case bloat slides including audience suggestions! Soon I plan to group and reply to the suggestions, but here are the highlights.
Most of the suggestions are in the actual paper, but unfortunately NOT so much in my talk as I ran out of time and there wasn't time for the participants to read my paper before the talk, so I'm pretty sure most of them didn't realize how much work and research I did before speaking on the topic or how much of this stuff I've done.
I learned that I forgot to mention several things:
1. We need a paper for how to prevent creating bloat when writing test cases. That is not this paper, but I agree, good topic! I'd like to see some data on having an expiration date set for test cases.
2. I had not included the "start over" idea because I assumed that your test cases had more value than THAT, but that was an assumption that may be very invalid. By all means, if you are dealing with test cases of little value, starting over may be the only way to proceed.
3. The person who suggested to cut any case they couldn't automate?? You scare me so much.
4. I didn't go into detail about "remove duplicates" because from my findings so far, this is not the lion's share of the bloat, but it is worth mentioning.
The most creative ideas I heard back were to prevent bloat in the first place.
Title This Speaker Proposal-And Peer Review Welcome [Testy Testy]
I'm submitting a speaker proposal for the Better Software Conference 2010 which is a biggie! It's my dream as a speaker to do CAST, and one of the STAR conferences, and this year I want to submit for both of them and do my best to work towards my goal. I've been in contact with the program chair for Better Software and got his feedback on the outline. I'd like to hone in my description and title and I'm very open to feedback. I feel this topic is really what I'm about at the core, and is the reason why I go through the work of presenting, so I know I can speak about this and love it.
Main Message: The code is one important source of data to inform abalanced test strategy, but it is not enough to focus only oncovering and improving the code. While many useful discoveries can be made with code focused testing, if we make sweeping assumptions that everything of value is in the code and limit our testing scope accordingly, many important defects may be missed. Test forwhat is missing from the code, for problems introduced by integration,and you can start finding the bugs that become apparent when you aretesting with a user focus.
Description: Testing the code alone is not a balanced or comprehensive teststrategy, and while it is important, is the user getting lost in all of the process improvements? If our goal is to raise to overall goal of the software, it is key to test for the user experience and not with only a code focused perspective. The code may be entirely covered and delivered on time, but if we fail to report the issues which impeded us deliver compelling software that humans enjoy using, our test coverage could benefit from expansion.
I. How We Test Today
A. Unit testing
B. Code coverage
C. Reviews and inspections.
D. Functional Automated Tests
E. Automated Regression Tests
F. Internationalization/Globalization Tests
G. Performance Benchmarks
H. Compatibility Tests (systems, browsers)
I. Beta programs
II. What Have We Missed?
A. The USER
B. Anything that is missing from the code
C. Integration problems
D. Requirement problems
III. Testing for the User
A. Data Points just waiting to be used
B. What is the code missing?
C. Integration testing.
D. Collaborative Workflow Testing
E. Scenario Testing
F. Usability-More than just “is that pixel right”
IV. Big Finish
A. Real examples of bugs that would be missed even with 100% code coverage.
B. Big Picture Quality-Focus on the code early can be useful, but it aloneis not a complete balanced strategy. Starting with reviewed and testedcode will give us a strong base to test from. When the code is covered,then what? Then you are ready to test for the user experience and useyour creativity and testing expertise to report bugs on anything thatdetracts from the overall quality of the software under test, if thatis in the code or perhaps should be.
Knowing the Code is Helpful, but NOT a Balanced Test Strategy [Testy Testy]
One of the testers who's writing and work I'm getting to know, Chris McMahon wrote about Testing Innocence. Wow-the link interface doesn't like this long of a link. Testing, what? Ok, workaround time-Copy/Paste the following longest link ever www.stickyminds.com/sitewide.asp?ObjectId=15405&Function=DETAILBROWSE&ObjectType=COLsqry=*Z(SM)*J(COL)*R(createdate)*K(colarchive)*F(~)*&sidx=3&sopp=10&sitewide.asp?sid=1sqry=*Z(SM)*J(COL)*R(createdate)*K(colarchive)*F(~)*&sidx=3&sopp=10
One thing that sucks about being a tester is I never get to USE software. I'm constantly finding bugs and I'm driven to report them to see them fixed. It annoys other people. If you collaborate with me, ask me to be a speaker, ask me to write something, I'll almost always report a bug or two about the software we are using. It isn't on purpose. I'm really trying to use the software, but I continue to have the tester eyes as after so long it is just part of who I am. I would say I've always been this way, but it has gotten worse the more time I've spent testing. After reading Chris' reasons why we should all read the code, I realized I have much to say on that, so I wrote the following.
Understanding how the code works is one good data point, but it is not
the only important data to consider. In fact, I would argue that what
is missing (or NOT in the code) is where the most important bugs lurk.
In my work I have had endless frustration trying to convince testers
and test managers that code coverage is NOT test coverage, that
functional testing is not "the testing" and that testing for the user
experience is vital and our automation isn't very good at it. My
concern is that rather than a balanced approach where the code is
reviewed and understood and considered as one important piece to data
that informs the ever evolving test strategy, it is seen as the ONLY
important input. I think understanding the code and testing from that
alone is far easier than testing the software with a balanced approach.
It requires less thinking. It is possible to be more innocent because
you can pretend "If it isn't in the code it doesn't exist, or if it
does exist it doesn't matter." If you focus your mind in only on the
code, the scope of anything seems more defined. It may be comforting, but because I love puns I must point out that denial is more than a river in Egypt.
First, let's look at what you mean by "understand the code" or "read the code":
1. You understand the design? Usually not. The code is not the design.
2. You understand the psuedocode. (You'd better! If you can't understand how it works, what are you testing?)
3. You can map/model the code. (Yes, I see this is critical).
4. You can write the psuedocode for the part of the product you are testing. (I think if you are pretty accurate here that would be a good exercise, but the map/model is more useful for testing)
5. You can read and have read all of the code for the entire system under test (Got a lifetime if you test Windows? LOL)
6. You can and have read the code changes and bug fixes that go into the product under test (Ok, this makes sense to me sortof-more than curling up with the code for an entire product does. I'm envisioning a long beard and flowing hair before you know one feature in the case of the products I work on).
7. You can write some useful code to test your product. (Yeah--I can see this matters.)
8. You can write code on your own product. (Hmmm. The lines between tester and developer are blurring. As much as I'm shamed to admit that my own dev counterpart fixed my Python because he needed it sooner than I could fix it, he helps me sometimes, why do I not help him? Well, I assume he wants the core product to work even sooner, so maybe I should? I still think if we went head to head at testing I'm winning in terms of important bugs found, but who knows? Maybe his understanding of the code would win? We may never find out.)
9. You are "on the bench" as a developer and your whole goal is to become a developer. (Why is it that some devs think we want to be them? I think some testers do have dev envy and want that job, but as I stated above, that isn't what I'm talented at. I like to find issues and see them fixed. I have natural talent for it. I'm faster and better at that than I am at coding. Like I said, for me, testing is natural, coding is painstaking for me. It's like digging a hole with a teaspoon and the whole thing is like a practice in patience. How long can I do it before I have a meltdown like a toddler. Reading someone elses' code? Much easier, but still not what I'm going to sit down with on a Sunday afternoon by the fire for fun.)
Now deemed as unimportant in the face of the almighty code is all
usage data that can be found (actual and user study information),
integration problems, user scenarios(real world and predictive), past
issues with the software(including top support issues), risks, code
changes (areas of activity), and prioritized environments along with
quality goals and company goals for the software. Is there anyone left
employed at this point who still thinks understanding the code isn't
important? I think all of those people have mostly been laid off at
this point. My question is, why haven't those who can't see or think
beyond the code been talked to about bringing more balance into their
testing. The code alone is not enough to test the overall quality of
the software. It is certainly an important part of a good test
strategy, but I think we are severely low on vitamin U and lots of
software sucks to use because of it. Yes, we need a user focus
supplement of Vitamin U and it needs to be better than just "let the
user test it for us and mark it as Beta for 3 years". Well, that
strategy has worked well for Google, but I don't think it will work
that well for all software companies.
I don't think my view is innocent or misinformed. In fact, only one who has
suffered the sugar crash of eating cotton candy for dinner could
appreciate such a boring thing as the need for a balanced testing diet.
So, I pose a question to you. How do you keep balance in your test
strategy while using the code? How do you keep the user in mind? How
does the code better help you create test cases?
While I'm not sure I agreed with all of these types, I got a great laugh out of these tester type descriptions, and especially the illustrations.
I'm prepping for PNSQC, stripping wallpaper and painting my laundry room, healing from surgery, caring for a sick cat, and working on two articles today. Wish me luck!
By the way, I am "The Networker" (and also started out as "The Magician". I had a 2 year stint as "The Expert"). People come to my office and ask, "Do you know who I might contact to find out about _____ and where can I find a chicken to sacrifice and a partridge in a pear tree? Oh, also, do you know where I can find a 5 year old build of ____ product?" I must admit I usually enthusiastically send them out with a list and apologize if there is anything I don't know where to find.
Process Improvement [Testy Testy]
If you read one thing today, read The Fishing Maturity Model post. While I have seen some fantastic process improvements, I've also seen some pretty bad ones. Personally, the best process improvement I've seen has been moving towards agile processes and collaborative testing and development under any name, but any way where you are vetting ideas together, reviewing eachother's code and planning, and improving and sharing knowledge together. For me the worst is when every little thing is scrutinized to the nth degree, over documented, and stingy experiments begin trying to save money without proof they will pay off but still they harm the software and the people. I call this Malevolent Tampering privately and wish for it to stop, but try to take a "we'll see" stance which means I consider it unknown and unproven if it will work well in the context it is being tried and would not make bets one way or the other. Anything which reminds you too much of "The Bobs" from Office Space? Beware the Fishing Maturity Model and start asking questions.
I'm gathering my thoughts and my courage to submit a proposal to present a StarWest 2010. I know. That is my dream presenter format. I would LOVE the chance to go.
So, if you've attended Star before or judged conference proposals, which topic would interest you more?
Code Coverage Goals are Reached! Now what?
or
Integration Testing Across Multiple Products
Which should I work on?
Python-possibly on a plane! [Testy Testy]
Friday a very awesome milestone was passed. My first working python project that applies directly to my job has been completed! I am working on making it better as in order to make it work I had to put in some information I can't share, namely password. I've since been made privy to getpass.getpass() by my boss (Go Go Smart Boss).
This week I need to stuff a good 5 days worth of work into 3.5 days as I found out today I'm having surgery on Friday. What was a nice appointment with the doctor to turn in a "log of symptoms" which showed severe symptoms on 14 out of 30 days of the month and plan a surgery for Christmas turned into surgery on Friday after I scared his patients by screaming during the exam. (oops) Apparently if you can't make it through the exam without screaming, waiting to 2010 isn't much of an option. So, enough of that noise (pun intended).
In good news, October is now full to the brim of testing activity! First into surgery, then into the glorious world of flying to Texas to see a dear friend marry and navigate the world of being a bridesmaid. Then I go home for 2 days and go directly to PNSQC!
I suppose before I go I'll update with a picture since I'm no longer at all scrawny. In fact, I can report myself a plump 5-11lbs overweight depending on the day, so pretty much American, yeah? Also, I haven't gotten a haircut in an entire year, so talk about redheaded. Major red hair for sure. So, my goals for the year in some ways are being downgraded. My goal to not get surgery all year? Not met, but pretty close. My goal to be at exactly a perfect goal weight? Not met, but pretty close. Too bad these aren't horseshoes goals.
There are some areas of life however where my goals are being exceeded. I am shocked to report that I LOVE python so far! It is 20% more Lanette compatible than any language tried since Action Script 3.0. I still curse when what I write doesn't work, but I don't curse and throw things, so that is great! And when it works? I love it! I have to show everyone which is pretty immature I suppose, but it results in great improvements to my programming and knowledge as I get feedback. I love my new job assignment! I'm having major chunks of the week where I get stuck in "the zone" and get bursts of work done. I'm learning really fast and I feel supported and appreciated in my learning by the team. It is my first agile project where I am on the team and have a status to share. I need to work on being more concise, but my enthusiasm is in contrast to brevity sometimes. If you read my blog you know why I don't twitter.
I suppose I have great respect for those who can paint ideas with one bristle, but I need a wall and a giant brush. Perhaps as I mature as a writer I can hone down my words to the essentials.
Gender Difference-Tell me about your team. [Testy Testy]
Last night at dinner Craig asked me to tell him about my new team.
I told him the name of each member of my team and something important about them, for example, one has a 10 year old boy who is in the Boy Scouts, one has an Australian Accent and runs marathons! I went through all the members on the team, and then told him I was excited to have a developer counterpart again on the job.
Craig laughed and asked why I told him all of this personal stuff when all he wanted to know was what the team does. Do we make software, or what? I then told him what I work on and what the team works on to answer the question, but I still think the most important part is what I told him first. What people are on the team and who are they really as humans? That's a vital part.
All these years getting in trouble for my MOUTH and this time I don't speak up! [Testy Testy]
I love this new job, but being Day 3 of "officially" having this new assignment I came in a bit early, day 2 of the smoke tests I insisted on, because we had this big "dry run" with the whole team present and lots of integrated parts. I found a bit of a problem, well, actually the smoke test failed. Ok, it wasn't a small problem. It was a MAJOR catastrophic problem.
So I emailed the team off of the chat room we were all in because I didn't want to shame anyone thinking that well, maybe what we have on the "dry run" setup is different than our staging server. I don't want to rock the boat since I'm already a newbie. So I send out the email, and 20 minutes go by, and I hear nothing. I join the conference room to "kick off" the process and they immediately ask me and the other QE if there are any issues. WELL! I'd sat on vital blocking information for 20 minutes. I was so afraid to cry wolf again that I didn't say a word to the public and no one had seen the email. So, I wasted everyone's time.
I've had strangers tell me that my mouth gets me in trouble. I just got fired from the HOA volunteer job I had for refusing to comply with a request from another board member that I didn't think was ethical and telling him that he could do whatever he wanted including fire me, but that I'd never comply with a request I didn't believe in and I wasn't going to cooperate until I felt I was doing the right thing. It turns out with the new job I wouldn't have had time for it anyways, so luckily for me I got fired. The point is, I don't like conflict, but I certainly don't run from it. I'm a big mouth and I'll stand up for what I believe in, when I'm pretty sure I'm right. That's the issue. I defer to those who know more. I'm just learning what I know and don't know on this job. I'm still digging through the log files to find out what is important and what is not. I'm still learning how everything connects and how to prioritize the bugs and tests. So this once I didn't speak up.
The last time such a thing happened, I turned to my best friend and sang, "God is mocking us, God is mocking us, God is mocking us,..from a distance." Now, she's a believer, both of God and of Bette Midler, so she understands I meant it in the best spirit, but even you atheists have got to admit, that's pretty funny when that happens. When the mouthiest person doesn't speak up, that's Murphy's Law.
Now next time I'm going to be all hysterical pre-maturely about a bug and it won't even impact the whole group and I'll look foolish.
So, today I learned the following: If you come in early to run the smoke test on a big day and it FAILS? Don't send email. Call people, tell the group, and send it with high priority status even if you risk looking like a fool. If it could waste the time of many people if you do not? Take the risk of looking foolish.
Think Small-Act Now [Testy Testy]
My wisdom of the day to you, in bitesized bits because I can only blog for 3 minutes while waiting for this critical bug regression test to run is to think small.
Why? Because when faced with a big challenge it is so easy to get overwhelmed. Do one small thing each day and it adds up.
There are so many things on this new job that we need to get done, but the first thing I started with was a small series of tests in the morning. Sanity, then smoke tests, then a tiny integration test. Once we establish a quality baseline at least we'll know if we are improving or not in quality. It is a tiny step towards what we need to get done, but it is something we can start now and build on.
It is better to have a small automated test that is working today than a brilliant framework that no one understands that is working 3 months from now. Think small.
Also while thinking small I minimized and shrank all windows of everything I was testing and reported tons of bugs. Then I changed my screen and font size. So, how else can you think small today for the sake of quality?
I don't mind if you tell me today that I'm small minded. If the tiny hat fits, the pinheads will wear it. Put one foot in front of the other and eventually we'll get somewhere sooner than those who are standing still waiting for a large brilliant plan.
I am elated to report that I started working in a new role in my company yesterday! I've been wanting to take on a new challenge and I'd been in the same job since 2006, so I've been trying out a few things. One of the project I've been helping out on, luckily the one I've enjoyed the very most, it turns out can use my help immediately and on an ongoing basis. I'll get to contribute to the overall test strategy as well as do actual testing on a mission critical service for the company! I'm delighted, enthused, pumped, and I love the team I'm working with.
So, while the good news is I'm taking on a new role and I'm learning so much and I'm loving it, the bad news is the reason why I've changed roles is this team really needs my help because they have been low on resources. That means lots of overtime, hard work, and a need for me to be productive instantly and to pick up depth of knowledge yesterday. Also, in addition to my Action Script 3, I'm dusting off my rusty but trusty batch file skills and trying to ramp up on some python for some automation we also needed to be up and running yesterday. Anyhow, I'm very happy to be taking on this new role and I have an opportunity to make a big difference.
PNSQC Preparation, Geek Girl Dinner, Test Case Bloat Dry Run, Birds of a Feather [Testy Testy]
Tonight I'll be heading over to the Geek Girl Dinner at Microsoft main campus. If you see me there, say hello!
This weekend I'll be working on some designs for the signs at PNSQC to help people get around during the conference. I hope they turn out well. Knowing my style, expect swirly, and a goth version of corporate.
Also, I'm teaming up with Jean Hartmann of Microsoft to host one of the Birds of a Feather tables at PNSQC on the topic of dealing with legacy test cases. He is also presenting, and her topic ties in with my topic, but his is much more from the automation side and explains how to reduce test cases using code coverage tools for more effeciency.
Last but not least, I'm working on completing my slides for the presentation since Jean has offered to arrange for me to come present at Microsoft prior to PNSQC! I'm hoping for great feedback, and if I'm able to get a recording of my presentation I'll share it here as well.
Counteracting Wise "Sayings that Say Nothing" [Testy Testy]
There are certain sayings that people use that seem wise, but really say nothing. They may be used to deflect blame, change the topic, or just fill up silence. Here are some strategies for working with some of these sayings.
1. Thank you, Captain Obvious. These are sayings that I even find myself blurting out on occasion and I wish I could retroactively shut myself up.
"It is what it is."
"What happened happened."
"Let's leave the past in the past."
"What's done is done."
"No use crying over spilled milk."
Reactions: No, do not say, "Thank you, Captain Obvious!" or something equally sarcastic. Instead just ignore it and maybe even a little play on words to et back on track such as, "Now what was it that happened?", "Ok, well, I'd like to understand what it is, now what can help us do that?" "Well it is past, but it is still impacting us today, so how can we make sure it is better in the future?", "How do we best mop up that spilled milk before it stinks and the baby starts crying?"
2. So what? These are sayings that if you think about them too long make you wonder why anyone thinks they are helpful.
"There is no "I" in team."
"We just need to do more with less."
"I'm 1000% sure!"
"Give 110% to get the job done!"
Reactions: "While there is no 'I' in team, there is a me, and there is also MEAT, but it isn't kind to eat your team or beat your team, so treat them well, all of them, and do not say this." "While I understand resources are constrained, what are the priorities so we know what to drop since 100% really is the total. There is no extra 10% nor is there a realistic way to do more with less, so what is most important to you?" Also, I've learned the larger the percentage of confidence, the more of a lie you are being told. The person who is 1000% sure really isn't sure at all. The person who is 95% sure is way more sure than any percentage above 100%. Do not trust people when they have a creative numerical scale.
3. Yeah, and, that changes the situation because?
"It worked on my machine!"
"That isn't ready to be tested yet."
"I noticed that, I just haven't fixed it yet."
"Oh, I already fixed that in a future build."
"No user is going to do that!"
"That doesn't happen in my file."
Reactions: "Let's find out what's different on my (machine, file, monitor) so that we can fix it." "When will that be ready for testing? Can I have a handoff when it is?" "Great, send the bug back to me with notes so I can regress around it when I get your fix." "Well, I'm a user and I did just do that, but if you don't think it is a common scenario, perhaps it's deferrable. Let's check with the user data to see what percentage of customers this is likely to impact."
The Superhero Tester-Here I Come to Save the Day! [Testy Testy]
We have this huge push like nearly any professional software company, to try to move our quality process further upstream and to reduce drama. This makes so much practical sense. Our senior director of quality is a super smart guy and he puts his money where his mouth is and puts down solid goals and really risks failure in order to change things. He isn't one of those people who 3 months later just gets distracted and stops talking about his previous goals. He's serious as can be about making less drama.
Well, I understand that goal but I can't say I find it emotionally appealing. The truth is that I started acting in plays at the age of 11. Back at Expo '86 in Vancouver Canada I was an elementary student and I had a choir, dancing, and acting part in a play at the World's Fair and I thought that was pretty cool. I directed one play and acted in 11 plays while in school. If you ask other people, "Do you consider Lanette to be low drama and laid back?" I'd be entirely shocked if anyone said yes. So, that leaves me conflicted. I do want less drama because I care about overall quality. That's for sure. I want predictable and excellent software. However, uncovering bugs in a crafty way, seeing them fixed, and knowing I saved someone from a nasty problem is where I get so much job satisfaction. I don't feel that way about just seeing a test pass. I feel pointless when I just do my part and don't see any result. I'm not fine being just another gear in the machine--pouring out reliable and steady automation that passes, or fails in an isolated fashion. I want corrupted documents, crashers, blue screens of death, and drama. I want the Young and the Restless of software right on my screen. I want a horror movie of database corruption and a torture movie full of performance problems to report. If you take all of the drama out of the software, then where is the fun and where is the excitement? I want to save us from the shame of delivering less than a worthy product because that is the really satisfying part of the job.
I'm working on the new job for Wonder Woman now, and I believe it's a good idea for me to work on this project. It's high pressure, high risk, and high drama. The difference between the testing and the real drama is that no one dies if we fail, and no one marries their wives sister due to amnesia in software (that I know of). I'm pretty sure if you are a test manager you have some testers who just want to be left alone to make nice reliable automation. Then you have your people who come in beeping and flapping about some bug or the other they just have to let you know about with that excited expression. Well, you have a tester with a super-hero complex on your hands. You can't put them on something entirely methodical and expect them derive remotely the same satisfaction they had when their life was a bit more exciting. Even with moving the quality upstream there are going to be some areas of risk. Let them sniff it out for you and get some excitement because otherwise your drama prone testers will start to get restless and possibly cause trouble. In fact, I know a guy who transitioned from software to fire fighting. Really. I mean for the fire department. I'm not a good accountant. I am a good fire fighter though. I'm good in a crisis. I'm trying to develop my skills that work well at all times and to become well rounded, prepared, mature, but that takes time. Right now, I'm still in need of some drama.
Load Testing:The team discussed it and we decided not to. [Testy Testy]
I'm working on this great team but I found out yesterday that the question of load testing came up, and the answer the team came up with was, "Thank you, no."
Huh? No?
I'm trying to phrase this correctly and instead of: Since we know it is so well designed that it is going to hold up, how about we throw twice our worst reasonable fears at it just once and if no issues are found at just twice the worst case scenario I promise the next day I'll come in wearing a t-shirt that says on it, "My paranoid tester thinking cost the company time and money. I made useless testing happen!" and on the back it can say, "I was WRONG, so wrong, and it isn't the first or last time either!"
I'm thinking a calm question of: Why did you decide it wasn't needed? is the more professional response.
Now I'm going to go home and have nightmares of people saying, "Testing? Yeah, we thought about that but we decided not to."
The top thing I love about this job is the products. I know everyone says it's the other people blah de blah. Fine, you be nice to people, and yes, we have great people, but for me you can find decent people many places if you are willing to listen and be selective. I'm here about the products. I consider myself defender and protector of products. The products here are quite good. Standout really. Worth fighting for quality in. Yes, it's likely I enjoy testing a bit too much.
Wonder Woman-No invisible jet [Testy Testy]
I have the amazing luck of spending part of my time right now working for this woman who I've wanted to get to know. She is warm and smart and has a way of accepting change that I could learn from. She also has this knack of being politically smart without being at all insincere or compromising or weak. I have no idea how to even start doing that. I really admire her.
So far I'm working really hard and enjoying her team. I'm trying to balance the old work I have to do with the new, but being me, I'm trending towards doing all of the work I want to do first and the other stuff I'm behind on. This makes for a sad realization on a Wednesday night that if I don't change my evil ways I'll be working too much over this holiday weekend and I sure don't need that. It appears I'll be stepping down a notch from project excitement. There are so many opportunities right now to learn and do that I can't seem to help but miss some of them. I'm just glad I didn't miss this one. Already I'm feeling pumped about it.
Exploratory Testing-Still effective despite the polish being off the apple. [Testy Testy]
Last night I told Craig about a bug I was glad to have found doing exploratory testing and how per the usual it makes me sad that it is undervalued as a test methodology. I told him I would try again to at least tell my boss that we are still finding quite important bugs with it.
Craig asked, "So, are these different people you are trying to convince that it is important, or the same ones?"
Me, "Oh. Yeah. Still the same ones."
I proceeded to think about it with a disappointed look. I've tried to not care about it, and it isn't that I disagree with using a variety of good quality practices at all, including well planned automation, using existing tools, and even creating tools sometimes. In fact, I'm involved in many parts of quality planning and execution. I just don't understand why having talent for exploratory testing isn't valued as one skill among many. It makes me sad because I'm not mediocre at it. I'm really fantastic at it and I love doing it sometimes.In fact, I quite enjoy doing other things as well. I just don't understand why the ROI on exploratory testing is so often ignored, considered too costly, or just skipped as being "the past", just as the ROI is inflated on automated testing when it is still missing so many bugs of critical importance and it is still so often sucking in the areas of sustainability and cost of adapting the automation to a changing product under test.
Reality. How much does it change when your perception of it does? According to "the Secret" all of it.
I'm planning to head over to see Harry Robinson speak tonight if my pain level is conducive to me sitting for that much of the day. There is no reservation status for "Living in a cage of pain. If walking with only a slight limp I'll be there, if grimacing and walking with severe limp signs point to no."
UPDATE: I drove over there and was so confused about why no one was there. Ummm DUH--It's September 9th!! Guess I hope that's a decent pain day because I'm excited about it.
I'd like to thank the good Dr. K and my wonderful and patient boss that I'm still employed at this time. I care very much about the success of the products I work on and the overall quality they produce. I put my talents to work in any way I can to reach the overall goals.
Talking about having good results with exploratory testing is sort of like talking about the importance of still testing what customers are using. It may be true, but it isn't an interesting story that people want to invest in. It doesn't sound good and no one wants to talk about it who has any power to finance more of it. It is something just taken for granted, like dinner being ready when you get home, where you don't comment on it or really notice until one day the suitcase is packed and you are eating takeout on the floor wondering why you didn't notice sooner that you had it pretty good. Because "it is being tested" no longer means anyone human is looking at it. It doesn't mean anyone has USED it or even evaluated it in a real scenario. I met an engineer I really respect in the lunch room yesterday who told me off the record about his concern that test coverage is so thin in terms of anyone looking at it with testing intent or even using the software anymore and he was commiserating with me mostly just worried about the end result. He isn't a lazy developer like some who just check in crappy code. He's one of the guys who even in the early days wasn't the cause of breaking the build even with complex changes. So, why is he sharing this concern with me in the lunchroom in near silence to all peers and higherups?
Because, exploratory testing needs to get back in the kitchen where it belongs so we can ignore it and take it for granted. We've got new exciting and unproven test methodologies to go on tumultuous first dates with. Only in desperation at the last minute when our new methods have failed to show up for us will we come back to the old and unexciting exploratory testing which has served us well in the past. There it is, our unstated backup plan. We call it, what worked before. It's our sure thing. Break out in case of emergency. There are a few of us here who are really good at that sort of thing and also can do other things. What about the day when there is no backup left? Perhaps I am no good because I don't move forward with wild abandon and easily forget about what worked in the past. I have an uncommon and strange sense of loyalty to my product, my customers, my team, and also to what has worked in the past. Moving forward for me means learning and adapting but only dropping the past entirely when it is no longer needed, not before. It has to be annoying though for a person who's already moved on to hear again from Testy Redhead, "Hello? This is still needed. Let's give Old Faithful another look?"
Guess I'll head to the kitchen now.
Whew! I've been working so hard that I've neglected my blog I'm sorry to say. However, please check the PNSQC 2009 schedule! You will notice that I'm on Tuesday, October 27th in the Testing track. The paper is final, but the slides are not, so I'll be working hard to make it a fantastic presentation.
I hope that you all are having a fantastic summer.
Learning ActionScript 3 [Testy Testy]
I had the pleasure of taking a class in person this week from David Gassner at BardoTech that made more difference than I could have predicted. He also offers some online training at lynda.com from what I understand, and I'll be checking that out. I have a breakthrough day on Wednesday and after class yesterday I got home and proudly told Craig all I thought I knew about programming and far more of it was accurate than ever before. The reason why is not just the length of the class, but the teacher and the other students. I wanted to share some of what worked for me in case you are trying to help someone else see some progress and finally gain some confidence in their code enough to not hate it.
1. Paraphrase constantly. Do not just say, "We are now typing addEventListener,...", instead say, "We need to know when a user clicks the mouse so that we can react. We have this class that comes with Flash that knows about all sorts of Mouse Events. So, we are going to be listening for that now.
2. Repeat, repeat, repeat. Not back to back, but at least 3 times in the day talk about the same key concept in different ways. Having multiple chances to understand really increases the odds to improve.
3. Explain WHY it is wrong when the student makes a mistake. Don't just say, "Well, you need a colon there, not a period." Instead of saying, "You need to move that code into the function above so it will work." you can instead talk about scope a little bit.
I am super excited because I went from everything looking like magic from the library to understanding about 40% of the basic examples (NOT complicated stuff) in 3 days time. It is the first time I enjoyed writting code. I am so excited to take a class on object oriented programming fundamentals because I feel like my weakness in that area is the most key factor that is holding me back. Once I get a little more understanding I don't improve a tiny bit, I improve in leaps. Any ideas of where I can take a good classroom Object Oriented Programming foundations course in the greater Seattle area? I want an in person course if possible as that is where I learn best.
I'm just so thankful today and wanted to share with you how much I loved my course! In other news, one of my co-workers has offered to think of an assignment for me to practice my ActionScript 3 with once I finish the 2 tutorials I have. Once I'm feeling more comfortable with writing my own custom classes and using what exists in Flash Authoring, I'm going to try learning Flex.
In other news, I'm mid-draft on my PNSQC paper and need to get the final revision turned in by Sunday. I'll be working my little fingers off this weekend taking breaks to meet Craig's sister who is in town from Kentucky who I've never met before.
Testing Nightmares [Testy Testy]
Several nights ago I awoke from a terrifying dream. I was in a classroom where they were training testers. Large banners of mission statements were hanging near the front and acronyms were posted around the room. The teacher upfront was sort of like an Amway spokesperson and would call out, "What are our A.B.C.'s?" This eager clean faced young Skippy replies, "Always Be Certain!" Everyone in the class agrees parroting back their learned acronyms. The guest speaker arrives and it was this developer I used to work with who I really respected, someone very smart but soft spoken. The slides come up and the presentation is a very technical and engineering focused slide show explaining nothing about testing in any way. As the presentation ends I start to ask questions about what testing risk do the changes introduce to try to get some discussion going about that. Silence and dirty looks. I try again and ask what development challenges were faced and get a vague answer. I give feedback that I'd like to learn more about the topic as it impacts testing, or at least follow up with someone who knows about the testing aspects of the technology and was looked at like I was crazy. Knowingly, the people looked at each other in the classroom. I was somehow shameful and an embarrassment.
I'm afraid that testing itself is becoming antiquated in the opinion of some people who are smart people, but simply do not see the point of testing beyond getting to complete code coverage. I worry that at the core there are some business people who do not understand that software testing has value because they are trying to engineer their way around doing any testing. Please note that this isn't about automated tools or test automation, because test automation when done well still should be about solving testing problems, and accomplishing tasks which result in higher software quality.
Assumptions that I fear:
1. Testing tasks can be done by anyone when using an incremental development model. Testing should be done by anyone on the team who has time in order to be efficient and "agile". In fact, by not having designated testers, you can possibly keep a faster velocity because everyone will do everything, including everyone coding on the project at the start of a sprint and everyone testing the features to get to done at the end of a sprint.
2. Testing tasks should be all done by developers. In fact, a CS degree and a testing certification are worth more than years of experience.
3. Testing qualifications are based on certification and coding expertise. Test knowledge is coding knowledge. Knowing how software is built in theory is far superior to testing because any non-code testing is "blind", "needlessly time consuming", and "manual" and is only done by the poor peons who do not know how to code well enough to use the better (automated) way of testing.
4. Good development methodology produces quality code without testing, therefore testing is obsolete as a practice and profession.
5. It is possible to move all testing upstream so far that it is no longer relevant to test completed and integrated and installed software in the environment it will be deployed on.
6. The ability to make good tools for other people to test with is equivalent to being good at testing as a profession.
7. While there is proof that it is cheaper to test than fix bugs once they have been released, if we just call our efforts for better development, such as unit tests and peer review and automated build verification and sanity tests our "testing efforts", then we have saved money and can safely assume no other testing is needed.
8. It is perfectly acceptable to sit in an office and write a pet project of code and not test because testing is not important, only coding is important. If the tool you are making is good enough other people will use it to test for you (what?), because there are so many testers left just waiting for another tool to do testing.
You know the most terrifying thing? I know that everything listed above is false. But why is it untrue? I'm aware that I can not PROVE that I know anything. I have no way to argue against those points and that is a failing of mine, not an incorrect assumption of someone else. I should be able to offer something besides my opinion and experience. Why don't I know a dozen references and studies and experts who can prove otherwise? Why is it acceptable that my knowledge has such a huge gap? I mean, I'm not new at testing, nor am I some person on the fringe who never reads anything published in the industry. I seek out information, yet I clearly don't know enough about testing and I have nightmares about it. The assumptions above can't be false just because I believe it to be false. The thing is, assumptions that are attractive that people are motivated to make because they save money in tight economic times do not need to be proven to be acted on. All it takes is a few words to a few people who are willing to believe it. A few whispers of "We don't need testing anymore. We've replaced it with robots and it's cheaper." to the right people, then what happens? If I am not prepared to stand up for testing with a well stated logical argument based on fact, I don't think I'm really good enough as a tester. I need to correct this.
I am looking for ideas, for science, for research, for proof that skilled human testing saves money. I believe that the majority of people in software today want to do what is best for the users while balancing out the difficult choices that need to be made in a tough economy to remain a viable business. While fear is irrational, I think it would be useful for me to have more facts and knowledge to back up my opinion that testing provides a better experience for the user. How does it save the company money long term? How can we measure how much and make sure that the value added is increasing as we improve our methodology and modernize our test practices? It is one thing to make sure that your test strategy is great at protecting user interests. Is it possible to also know that your testing is effectively contributing to lowering costs and providing value for shareholders? For projects that are for a "for profit" company, these questions are important and fair to ask. My nightmare may be that testing goes away and is replaced with nothing but smoke and mirrors, but to those higher up in the company, perhaps their nightmare is that the company is gone in a puff of smoke. Balancing the needs of the users with the needs of the company can be tricky and difficult, but I think that it is one area of test leadership that I don't know enough about. In order to grow I should have more knowledge and practice and understanding in this area. In the past I've avoided it because I felt that everyone always wanted to do what was best for the user, because what was best for the user was of course best for the company.
As a test leader, what ways do you express the value that testing adds to the stakeholders who are most interested in the results of testing based on money savings? How do you measure the ways that testing saves the company money?
User Experience Matters [Testy Testy]
I've been putting off getting a new phone for many reasons. The last time I got a phone was 2 years ago and I got so angry about a text messaging bug that I got in trouble with Craig for cursing and talking bad about Verizon. Craig even argued with me that it isn't a bug, but I told him they had more than 5 choices of how to handle this issue and they chose the 1 worst possible answer for the user. Here is the situation.
When you send a text message from a Razr to any non-verizon phone which has more than 160 characters the additional characters would not fit into one message. What that results in is a series of choices. Either Verizon could restrict the entry to 160 characters if they KNEW the message was going to a non-verizon number (but that would require more work to actually know in advance what network the message was going to), they could alert the user before sending that the message would be lost after 160 characters IF anyone they are sending to was not on Verizon. They could give the user a choice of sending the rest in a new message before sending. They could offer a "settings" option where in this situation a user could set the default to send 2 text messages in this case. Worse, but still useful, they could save the rest of the characters in a draft message if it couldn't be sent. But no. Here is what they did.
They give the user NO alert, but once you sent the message they text you back immediately that you MAY have lost the rest of the message without specifying WHICH recipient wasn't on Verizon. You lose all of the remaining characters, so have no way to send them again. This has resulted in partial freaking addresses. Unintended meaning in sentences. Jokes with no punchline. Frankly it makes me so mad when I hear that sound of an immediate text message coming back that it LOST my message and I have no idea who would get the rest even if it bothered to save it for me (which it didn't) that I hate my phone, I hate Verizon, and I don't want to buy anything from them again even though the reception for phone calls was better than any other network. What Verizon fails to understand is that I don't like to talk on the phone. I like email and text messages. They made a terrible decision on ONE common situation and failed to make a reasonable option for a texter who can get verbose.
Yesterday I bought an iPhone. WOW. I am blown away. I feel so bad for Craig. He bought a Fuse. I experienced absolute wonder and delight discovering my iPhone features. He discovered frustration and disappointment with intermittent excitement. The difference? It's all the UI and the experience. I can not believe how attached I already am to my iPhone. Now I get it. It is so fun to play with. People will pay for an amazing experience. That's all there is to it. People don't emotionally care at all about the logical reasons why. Do they WANT to use it? That's the difference. That is why the iPhone is smoking all of the other so called "smart" phones. You'd think people who know advertising and marketting would get it. Sexy and fun and "smart enough" always trumps "super smart but kinda dumpy and dull" when it comes to popularity. It is just built in that we love eye candy and it makes us feel good. So, in short, the iPhone is so amazing that it doesn't even feel at all like a phone. It feels like a miracle. It feels delightful. It feels like a privledge to use. I'm blown away in love with this gadget. I love it like I love my Roomba. That much. Good luck prying it out of my hands.
Trig for measuring gutters from the ground [Testy Testy]
You will be happy to know that our house is 17' tall. We first measured with the laser. I then got the rickety old ladder out and measured by flinging the tape to the gutter and measuring from there. I have video of the laser remeasure to share with you and the math.
After the measuring was complete we watched a TV lecture about robotic cars. I brought in the gutter cleaning robot for Craig to cuddle up to while we watched it. I have such a fortunate and geeky life! 
I just wanted to share. I'm writing like mad to finish up draft 1 of my PNSQC paper. I hope the reviewers like it. I'm really open to input and I just want so much to present and discuss this topic with others. I'm considering doing the poster paper just so I can hear the ideas others have on it.
Measure gutters with laser.
http://blog.testyredhead.com/2009/06/20/trig-for-measuring-gutters-from-the-ground.aspx FYI-you should have seen Craig's irritated expression when I told him that 5 and 1/8th inch was the same as 5.18 inches. It really is too easy to tease him.
I just wanted to share. I'm writing like mad to finish up draft 1 of my PNSQC paper. I hope the reviewers like it. I'm really open to input and I just want so much to present and discuss this topic with others. I'm considering doing the poster paper just so I can hear the ideas others have on it.
Measure gutters with laser.
Gutter Cleaning with Engineers [Testy Testy]
I share this with you because as a tester, one of the things I love the most about my job is working with other geeky people. Now that I have my very own dev counterpart at home, I am also amused with how engineers approach problems other than just customer problems, software problems, and bugs.
Mission: Clean the Gutters
My Idea: Hire someone.
Craig's Response: Too expensive, besides, needs to be done often. Let's do this ourselves.
My Next Move-Buy
adjustable ladder which claims to be the "only ladder we need" on sale
for $99. Assumes this is absolutely good enough for the gutters.
Craig-Reads
reviews, watches online performance, and then buys gutter cleaning
robot from Roomba to automate this task while negotiating half payment
of robot by me.
Testy Redhead-Gets robot in the mail, reads all instructions, puts it together and charges it.
Craig-Finds ladder not tall enough to reach gutters in some areas of the house.
Both of us last weekend-Go seeking extension ladders. Even the $200
ladder holds only up to 200lbs WITH all supplies including the ladder
itself and the robot. Realizing this means I now have to do ALL
tasks on this ladder, we refuse to buy it as to be safe with the ladder
itself being 33lbs anyone over 167lbs shouldn't be climbing the ladder
according to Craig, which rules him out even if I don't feed him for
quite some time.
Testy Redhead-Sends Craigslist links to Craig for
ladders that are taller, cost less, and hold enough weight for Craig to
climb it with the robot.
Craig-Being unsure how tall we need the ladder to be determines we need to know maximum height.
Testy Redhead-Suggests we get a tape measure and climb our current ladder to measure the difference.
Craig-Shoots
down this idea because there is no way that will work and my tape
measure will flop over (I don't think it will, but he does).
Testy Redhead-Suggest
we tie a rock to a long piece of her knitting yarn, throw it into the
gutter, straighten it to the floor, mark it off and measure the string.
Craig:
I've seen you throw and we have windows. I don't like the idea of you
hurling rocks at the house. This could end very badly. I have a better
idea.
Craig's Brilliant Idea (that we are about to go do): Measure the height of the gutters using a laser pointer and the Pythagorean Theorem.
We'll measure the distance we stand from the house and the angle it
takes to get the laser to the top of the gutter. I kid you not.
My reaction to this was to laugh hard for about 4 minutes, and finally say, "I'm so blogging this."