I have had the opportunity to review a lot of automation test code and to work with teams to begin peer reviewing each others code. I felt it would be useful to codify and share some of the criteria I use. And just to be clear, this criteria is what I use for the automated User Interface and Acceptance tests for web applications. Many of this may apply to other test types or system, but no warrantees are expressed or implied if used outside of its intended use. Continue reading “Code Review Criteria”
What is the most offensive word that you know? Would you use it at work with your colleagues or with your manager or team lead? Maybe you would under certain circumstances. How do you react when you hear your colleagues using it around you? I certainly don’t mind a certain amount of swearing, and use it myself from time to time. But there is one word that I really can not stand. And after having heard it used on me four times one day recently, I just about lost it.
So what offensive word am I referring to? The word is “can’t“. That is right, can’t is one of the most offensive words that I know. And using it with me may result in an unpleasant response. Unfortunately I hear it way to often in a business setting. Continue reading “Can’t is a 4 Letter Word”
In a previous article I discussed how to prioritize a test automation backlog. An important part of prioritizing and tackling a test automation backlog is to identify or calculate the business value of the each of the tests.
Calculating the business value for an automated test is similar, but somewhat different, to calculating value for developing a product feature. The business value for building test automation for a particular feature is calculated based on four factors. The first is the percentage of affected site visitors for the feature under test. This initial value may then be modified for the financial impact of the feature. Next the value may be further modified to account for the probability of failure. The last factor to consider is the incremental impact for the feature. These are each defined in more detail below.Continue reading “Test Automation: Calculating Business Value”
Starting a test automation strategy may seem daunting due to the very large number of features and use cases that will be identified for automation. Determining where to start and the order to tackle the backlog of hundreds or thousands of potential tests can be paralyzing. I recommend three techniques to organize the test automation backlog. First, should this test be automated at all? Second, what is the business value of this test? Third, what is the difficulty or effort to implement this test?
Remove any test from the backlog that should not be automated. Then use the business value and difficulty/effort metrics to prioritize the backlog. Start with the highest value, lowest effort tests first and schedule them to be worked on. I would not worry about scheduling out more tests at this point. When you have worked through the first list of scheduled tests, your should reevaluate your backlog again and schedule the highest value, lowest effort tests. You will find that you will have added items, or that the business value or effort will have changed since your first reviewed the list, so reevaluation is important.
When you review your prioritized list and see items that have a higher effort then business value, these should not be worked on. These would seem to be an example of tests that should not be automated at all. After all, if the effort exceeds the value, then they would clearly be a poor choose for automation. However I don’t suggest that you simply remove these from your unscheduled backlog. The reason is that over time the effort to automate some of your more difficult tests will probably drop. Your tools and frameworks, as well as your teams growing skills, will reduce the effort for many of these tests over time. Also the effort may decrease as a side effect of implementing higher value tests. Review these items along with the other items on your backlog periodically.
Calculating the effort to implement a test is something that most automation engineers will grasp fairly easily. Depending on the team, this may be in story points, hours, to-shirt sizes, or some other technique. However calculating business value and identifying tests that should not be automated (outside of business value and effort) is not so straightforward. I will cover these in detail in future articles.
Programmatically controlling a Sauce Connect Tunnel
I recently decided that I wanted to create and control Sauce Connect tunnels from within my own test code and could not find any examples of how this could be done. After some research and experimentation, I was successful and so I wanted to document the method that I am using in case anyone else finds a need to do the same. Note that my test framework is written in Java and the method that I outline should work with any JVM language. Modifying this to work with another setup is left as an exercise to the reader. Continue reading “Controlling Sauce Connect”
To start with lets look at the several different methods used by various programming languages to handle parameter passing. The two most common are Pass-By-Value and Pass-By-Reference. There are some other less common methods such as Pass-By-Name that I have used, but we’ll skip over those since the are not applicable here.
In the Pass-By-Value method, parameter variables are copied onto the stack and then made available to the function. Once the function returns, and the stack is popped, then the copies are gone. This means that the the variables in the outer scope will never be modified, no matter what the function my do to them. This provides a nice safety net to ensure that a function does not accidentally modify something that it shouldn’t.
In the Pass-By-Reference method, a reference (pointer) to the variable is created, placed on the stack, and made available to the function. Once the function returns, and the stack is popped, then the reference is gone. However unlike in the Pass-By-Value method, any changes made to the referenced variable will be visible in the outer scope. This can be more performant when dealing with large data structures since you do not have to create copies of the entire value.
I attended the CodeStock development conference in Knoxville recently. One of the sessions that I attended was the Woman in Technology panel. Michael Neel was on the panel and he made an interesting comment. He stated that while there were woman in technology companies, woman did not give keynote addresses at technology conferences. Continue reading “Woman in Technology”
I have been attending CodeStock 2014 here in Knoxville the last few days with a group of my software and quality assurance engineers. Several were even speakers this year! Thanks to the conference organizers for putting together another great conference this year! Continue reading “CodeStock 2014”
I will be speaking at the Agile Knoxville meeting on Aug 6th. See their website for details. The topic will be Distributed Scrum. It should be a good time.
The Agile Knoxville session was really great. There was a good set of people. Thank you to everyone that attended, you all had good questions and feedback. And thank you to Adrian for forcing me to come speak. 🙂
I was invited to speak at the Agile Development Conference in November in Orlando. It was a great experience and I’m glad I agreed to do it. I have received some very positive feedback from some of the attendees at my talk.
My talk was entitled “Distributed Scrum: Dangerous Waters – Be Prepared”. See my talk page for more details about the conference and my talk.
I want to extend my thanks to the conference organizers that invited me and for providing guidance. I also want to thank the members of Scripps Networks communications and marketing departments for their assistance with my presentation and for providing company swag to give out. Thank you!
Finally I also want to thank the people that showed up for my talk. Thank you for showing up! I hope you found something in my talk useful.