Reality

Last week was the grand finale of our course: guest lectures. We did not make these lectures mandatory for the participants of the course but I was hoping that everyone would attend. I have to say that those who did not missed a lot. And I do mean a lot. The lectures were perfect, a true grand finale for the course.

On Tuesday we were given insight about challenges related to testing cross-platform products in one nameless company. We were basically educated how they are organizing their testing activities and what are the biggest challenges they are facing. The lecture lasted almost two hours and was packed with huge amount of facts, solutions, problems, etc. What was surprising to me was the traditional "QA department" approach and lack of test automation. Still it is evident that the approach works for them.

I must say that I really enjoyed the presentation. No marketing b******t, just facts. No "we are succeeding in every area", instead successes and challeges were both presented. And the lecturer had so honest, down-to-earth style that I just loved listening to him. So I would give full 10 points for the lecture.

Thursday was dedicated to a lecture on tester's life. What is it like to work as a tester? Based on the lecture: challenging and fun. Again the lecture was perfect. Just perfect. The lecturer had prepared a lot of material and we did not have time to go through all of them but still many things were discussed. Exploratory testing, challenges when working with clients, variety of job descriptions for testers, what is actually testing etc. We also learned how to get over mistakes.

Again the session had good down-to-earth vibe and people attended by asking questions. I liked both the ideas presented and the approach to subject. So again full 10 points. I really hope to see that guy in action again some day.

Then some complaining. We have a lecturer from a company listed as one of the hottest in the business today. You would expect rush and excitement. You get approximately half of the course participants and handful of other people. We have a lecturer giving an insight on what it actually is like to work out there in the software testing business. You would expect all the course participants sitting eagerly in the room asking questions. And numerous other interested people. You get even fewer people than in the first lecture. That's a bit sad.

P.S. If you are interested on the lecture materials, you can get them from me.

Sessions

The fourth, and the last, dojo was not actually dojo at all. Reason for this arrangement was that one of the things we were supposed to study was sharing and organizing testing work. The easiest way for practising this was to let the groups self-organize their work using some of the ideas given in the theory part of the session. For me it seems that we didn't quite achieve the goal. Again, the concept would have needed some fine tuning.

Last time we realized that we needed some means to organize our testing efforts so that we would be able to answer some tough questions from the business side, divide our work among fellow testers and see where we are at when it comes to course of testing. So now was the perfect time to introduce some easy ways to organize notes related to testing (mindmaps, simple templates, SessionTester and RapidReporter). Also along with Session-based testing we were given even more ideas like timeboxing, charters/missions, testing metrics and debriefing.

Presentations were held and again people had done good work. In the Session-based testing presentation I especially enjoyed the small discussions on metrics and agility of the method. To summarize, I believe we ended up realizing that utilizing all the original Session-based testing things strictly might not be a good idea. And that everything related to metrics should be handled with care.

The tool topic was again structured nicely: emphasis was on demonstrating the tools. Although the latter of the two demos lasted a bit too long. I hope at least one thing related to any tool came clear based on the whole session: tools should be as simple as possible. I'm glad that people chose differently from the given notetaking methods when getting their hands dirty. Again, tools should always be chosen based on need coming from inside. Not forced by someone from outside. In this session people simply chose one serving their purpose best.

This time we were split into two groups a bit differently. Both groups were given different challenges and they were supposed to work as pairs, dividing the testing work related to their challenge so that they could get maximum coverage with minimum overlapping work based on testing strategy they chose. The Android team started their work wisely: the pairs were all given some time to get familiar with the product under test before starting to create the charters. Also the Utu team got their work going nicely and made a clever top-level decision on their tactic.

Things evolved without any big struggle so we decided to disturb the testing work a bit. After lunch the groups switched challenges and tried to continue work of the original groups based on their notes. This part was something that perhaps should've been instructed a bit differently. The idea was great, but the results of the switch could have been even better.

The one thing I was disappointed most in this session was the debriefing. We decided to leave the bugs out of the discussion and just tried to get some progress/metrics related things out of the groups. My questions were bad, they should have been more precise. Or we should have required more "official" summary from the team. Still, I hope the actual notetaking, designing charters, dividing the work and trying to figure out what you are supposed to do from other people's notes were something that made the session beneficial. At least based on the feedback students were still quite happy on the session.

Guest lectures still to go. Hopefully most of the students will come there. Every activity counts when grading...

Exploratory Testing

The second to last session on our journey to know what testing really was was about exploratory testing. The presentations were really good. I learned a lot and to be honest I actually expected a lot from the presentations.
One of the one part I am feeling the least of confidence in software engineering is exploratory testing. Mostly because I haven't been really doing that. I have high hopes for the future and maybe, maybe I could use some of the learning in my current job. Maybe I could participate ET sessions where the whole team is doing that. Maybe there's something new for me next Friday.
I did not do extensive preparations to the 3rd dojo. More or less I came with all I had. No excuses about not having time - rather I prioritized something else more. Well, I did join to learn. And learn I did, that's sure.

As mentioned in the session, I haven't used any bug tracking software for ages now. I think the last time we tracked bugs in a tool was early 2007. That is, either we haven't had any (well, we did - in fact 2) or we have been tracking the bugs with orange index cards on the wall. Either way, seeing an open source bug tracking software (let alone project management) in action was a thrill.
That would be pretty much the kind of software I'd like to use. And I've heard quite a bit good rumors about Redmine. So, a tool well worth a study.
When it comes to the dojo, our decision of using the university web site as our problem context seemed to be a rather good call. The instant hassle after we revealed the big secret really meant that we selected a site that arouses interest. How cool would it be to find a bug in our university web site?
We took a deliberate *risk* in exposing the whole university web site for testing. Yes, it is a vast area. Bigger than most testing task ever will be. The groups ended up taking two different approaches to the challenge. All in all, testing turned up being somewhat random, or could I say even ad-hoc. No worries, it was pretty much what was expected. And the most important part was that it was fun.
About Feedback:
We were given feedback that the challenge was too big. That we would've learnt the same lessons in smaller sites. Yes, we agree. And we also see a lot of benefits in randomly going through the selected site. First and foremost it seemed to be fun. And, with no less of importance, we hopefully all realize that ET is not something that can be done with no planning. Just starting to 'do ET'  does not make it necessarily exploratory. It is hard and it will be even more fun when getting 'into the zone' after some preparations.

So, the ad-hoc type hassle trying things here and there was pretty much what we expected. With no clear guidance from us. Nearly none, to be exact. And as said, it was a deliberate decision. Seeing what testing could end up being (ad-hoc) gives us (or rather, you) a great chance to introduce a more meaningful, planned and coherent way of doing ET. I am hoping that we are building a story of ET (from ad-hoc to planned) just like we did with ATDD.

Let's see how things go!

Exploring

Moving on to manual testing. With an exploratory twist. How to teach a subject that may seem a bit abstract and is mostly about using your own brains? That is the question we had in mind when planning the first Exploratory Testing dojo. We ended up choosing a testing challenge that was obscure and broad but what we thought would be motivational and fun for the students. The fun part was a success, both groups seemed to really enjoy the bug hunting. I guess the challenge was motivational too because people really tried to find ways to weak spots in the website under test.

Presentations were again given and they were good. We decided to skip the Redmine part in the dojo, to keep the focus on ET. Still I would say that all students were given a good example of a bug report and a tool that could be used in tracking bugs. For a tester bugs belong to daily work so an introduction to reporting is a minimum in a testing course. One of the groups already reported a bug they found so they also had a glimpse on that.

The presenters and we gave a couple of "brain tools" for the groups before the actual challenge. It was interesting to see that both groups chose different approaches. The one I was in decided to hit the probable weak spots in the website. That was mostly forms, and translations also when they realized some problems in there. The other group chose to use some role heuristics. Both groups found plenty of small issues in translations, some logical problems, usability issues, expired certificates and also goldmines for spammers. So a lot of things in an application that is up and running. And well tested.

Prize for the groups were plus one to grades if they find and report a bug. My group has already reported one so they earned their prize. The other group had a plenty to choose from also, so most probably they will get their prize also.

We were a bit afraid that the session would be a big jumble. For my group I have to say it almost was. Still I would say that the end result was on the positive side: people had fun and used their brains to challenge the system under the test. That is testing, the rest is getting more professional. We will look into that in the next dojo when we try to find some ways to move from ad-hoc to more efficient, organized and trackable working methods.

Although I am really proud how things went on Friday some fine tuning on the challenge should have been made. If I use the same challenge again I will make it more clear and present it in a bit different way. Also the analysis part would look a bit different if I do it again. The good thing was that we went through some findings and also realized together that there were a lot of things we need to do differently to fulfill all the needs for testing. More on that next Friday...

Watir, Cucumber and some random thoughts

The day was really good. Apart from feeling sick. And being sick.

For me it seems that we have done something right. Seems that you have learned some valuable lessons. We all have. What I learned together with the group contains

• how important it is to close the browser after each test case.
• how important it would be to clean up the database after each test case.

That is, there is need for setting up the context and restoring the system back to known state.

And what's even better - we found out that we needed some more information to the web page, namingly ID tags for certain entries. And then we went to the programmer who implemented it. As if the programmer had not thought the importance of identifying objects in the page.

Hearing comments like these, althought hidden behind the words, makes me to think that there has been a lot of learning.

One of the random thoughts I heard and liked the most was something along the lines that 'we are making the test pass and then we are ready.' Interesting thought. And it seems like that. And it could be like that. No, wait. It should not be like that.

If we are implementing an example and make it pass, how do we know that it verifies the right thing? What if something changes and the example does not highlight the change?

No. When implementing a test (let it be executable specification, unit test, or any scripted test) we have to make sure to make it fail before moving to the next test. And to fail in a way we know that it is failing for the right reason. There is a reason for having 'make it red' in the famous TDD cycle. And it is all that important for ATDD/BDD cycle.

As the saying goes: "make it red, make it green, make it clean". We did end up working on all phases. Good for us!

So, thank you all for speaking your thoughts out loud! And for the learning experience we had together.

To not pollute this blog entry about technical details, I will be posting my learnings of our use of Watir in my next blog post.

So that's why all these feature things were written

Journey continued last Friday. We spent 4 hours exploring UI test automation. Presentations were again given, this time only one per topic for the whole audience. Some discussion was held but we still got to see only a glimpse of the problems and solutions related to automating UI tests. Especially the design patterns used among at least some test coders would have been something interesting to try in the dojo. But we didn't get that far.

Tool topic was about Watir. I really enjoyed the approach in the learning session. That is, the group had prepared some simple example codes and went through them. A bit more clear and thorough explaining of the code and brief introduction on the helpful links (examples, cheat sheets, documentation) would have made the session perfect.

The actual coding part was a success. It looked like the session cleared many things related to previous dojo. Nice. The group I was in concentrated again on the problem, really tried to push towards a good solution and used the experts to get over difficult parts. The task was a really simple one but we already saw some concrete things that are present always when you write UI test code. Like how to verify, what to actually test, how to do it DRY enough etc. Small things. But valuable.

Then the improving part. Based on the feedback some attendees found the working method we used too slow. When combined to unfamiliar language (Ruby) the achievements were not enough for some. What then to do about it? Dojos are slow in nature but of course things can be done to make learning smoother. Perhaps we should have encouraged people to learn some Ruby before the dojos so we could have been able to skip the things related to language syntax. I find the hiccups related to unfamiliar language still quite small. Also a longer dojo (=coding part) might have helped. Or then we could have only introduced the things in the dojo and let people continue coding on their own.

So a few things to consider. One thing I did in our group was that I "easened up" the rules and allowed more talking from the watchers. Just to get over the language related things. But next time will try to concentrate more on that.

Next Friday we will then concentrate on whole new things. Exciting challenge is waiting for us there. Different but exciting. 

Learning Diary of a Brave

Check out this learning diary made by one of you:
http://jarkjar.blogspot.com/

Great job! Please visit and post a comment. If you dare...

Dojo subjects

As a reminder, the schedule and subjects on rest of dojos:

CodingDojo2: Friday 25.11.2011, 10 - 14 UI Test Automation, Watir
TestingDojo1: Friday 02.12.2011, 10 - 14 Exploratory Testing of a Web application, Bug management with Redmine
TestingDojo2: Friday 09.12.2011, 10 - 14 Session-based Testing, Reporting a Test Session with RapidReporter and SessionTester

And don't forget the guest lectures on :
Tuesday: 13.12.2011, 10 - 12
Thursday: 15.12.2011, 10 - 12

Features may look different

 Like promised, here's my suggestion to the "Create a new player" feature. As you can see the format of the file may differ but the relevant scenarios should be there. Remember also that both groups had different acceptance criteria. That means that the automated tests and documented behaviours should be different.

-------------------------------------

Feature: Create a new player

In order to be able to start managing players
As a system administrator
I can add players to the system

Scenario: Adding a player
    When I add a player 'John' 'Doe'
    Then a player with name 'John' 'Doe' is created
     And the player has default picture

Scenario: Adding a player with picture
    When I add a player with picture 'player_picture.png'
    Then a player with picture 'player_picture.png' is created

Scenario: Adding a player with a name of an existing player
    Given there is a player 'John' 'Doe'
     When I add a player 'John' 'Doe'
     Then there are 2 players with the name 'John Doe'

Scenario Outline: Adding a player without name
    When I add a player '<first name>' '<last name>'
    Then user is given an error message

    Examples:
    | first name    | last name |
    |                     | Rambo      |
    | John             |                  |
    |                     |                  |

Scenario: Adding a player with invalid picture
    When I add a player with picture 'invalid_picture.tiff'
    Then user is given an error message

Well done, all of us

Hi all!

My first blog post ever. Not counting Blipfoto as a blog, though.

I've had really interesting weeks. Trying to complete the Rails web example for the dojos and for our use. Not quite getting it ready enough. Well, luckily there's still few more nights to come. And maybe I will end up squeezing a bit more functionality to the app.

I've learned a lot. Ruby on Rails for starters. And I've never been doing any web front-end development. But I have to say that it's been a blast - coding in the living room some nights. Being really tired at work. And I mean really tired.

Some one tweeted the other day: "Coding for fun... and even learning while doing it!" I know the feeling, but I don't see any news in there - when positive feelings are part of anything we do, we tend to learn. And learn a lot. Thus, coding for fun does lead to learning.

Testing for fun does lead to learning.

A bit of-topic. My mind makes these tricks occasionally.

I really liked the dojo we had. For me it seemed that we concentrated on the most important topics I see - the communication between the developers (note, as developers I mean both testers and programmers) and the business. And you did find many aspects that should be verified. If not in executable specifications, then in exploratory testing. Some examples could be
* player named "; drop tables;"
* player with really long names
* player with some strange characters, like a space bar.

And as what was the best part of the discussion, you did suggest an improvement - what if we'd have a username. Then we would not need to identify two persons with same names from the pictures. Pictures which could both be the default ones.

Hmm, why on earth did that not come into our minds? And also trying to implement features in 'wrong order' - how stupid would that be?

I promised to implement the features myself and to post them to the blog. I did change the feature files to require username - as you suggested us to have. Thanks, it was a good call!

The feature file as I wrote it is as follows:


Feature: Managing players
In order to add, modify and remove players in the system
As a receptionist / person in the back office
I need to be able to add, remove and update player information.
So that Players are never removed, rather set inactive

Scenario: Add a player
When I add a new player
Then the player exists

Scenario: mandatory fields
When I am creating a new player
Then following fields are mandatory
| field |
| first_name |
| last_name |
| username |
# And the user will be informed about the failure

Scenario: Player has a default picture if no picture given
When I add a new player without a specific picture
Then the player uses the default picture

Scenario: upload a picture
When I add a new player and upload a specific picture
Then the player exists
And the player has the picture

@wip
Scenario: Removing a player makes it inactive
Given a player
When I remove the player
Then the player is not seen as active player
But the player exists

@wip
Scenario: Listing players
Given a bunch of players
When I list the players
Then I see all players
And they are in alphabetical order by last name - first name.

@wip
Scenario: Player listing shows picture and name
Given a bunch of players
When I list the players
Then I see a thumbnail of the picture of each player
And their names



All those are currently executed directly on the model/controller using the 'Direct Model Access' that's often used in Rails tests. On Friday we'll see how we can execute some of those using the UI. The scenarios tagged as 'wip' will be executed using Capybara simulated browser - yet another means to make testing faster than using a real browser.

But more about that on Friday.

By the way - I'm missing at least one scenario. What could it be?

As last thought few words about feedback. I kind-of totally forgot to explicitly thank for the presentations last Friday. They were excellent. I mean really excellent. You seemed to really worked a lot to get all the necessary information so that you could answer to the questions your student friends (and I) would be asking. Excellent job!

As what comes to our feedback, I hope that we will make a use of a 'feedback door' in the lectures to get course feedback. The most important factors being 'how beneficial the dojo was - scale from 1-5' and 'open comments / free word'.
And more about that too on Friday.

Interesting day coming. I'm so waiting for it.

Dojoing

The first dojo was a success. There was some of uncertainty still in the air on Thursday but luckily the black clouds stayed away. Experts were there and gave good presentations. They also kept their role all the time which was a nice surprise. All attendees were able to keep their attention the whole session and seemed to focus on trying to solve the problem. I really believe that some of them really learned something. So mission accomplished.

A few more positive things. Questions were good and on the topic. Some of the attendees also stated their opinions on different subjects. When the negative things were enquired we already got some things where we can improve next Friday. So next time only one presentation at a time. And more noise from the people typing on the machine.

For me the session was easy. I tried to give a couple of improvement ideas and guide towards good solution. The feature file we created still needed some fine tuning after the session but still all the relevant things were there. Not bad.

A couple of details. My answer to the "how about the regression and ATDD cycle" question was bad. I should have just tried to explain that the cycle is about getting new features done. The idea on automating the acceptance criterias and working (in small) using TDD then works towards catching regression automatically. When some bugs still get into the product they are then handled according to project's/team's/companies processes. And please note also that this is only my interpretation...

I should have also spent some more time on the business case and purpose of the feature. The "In order to... As a ... I can..." part of the feature. That is an important thing when you think about the feature although not that visible anymore in the implementation phase of it.

Hopefully next time we will also get some nice discussion in addition to actual results. Looking forward to next Friday. Will still create the feature file like promised.

First things first

So the course started on Monday. I had prepared an amateurish slide show about the course arrangements. On Saturday the plan was still to talk on the course arrangements only for 15 minutes but soon I realized this is the topic we should concentrate. Because the dojos and group works are the most important things in the course. So we spent 45 mins talking about those subjects, but still some things were left unclear. I should have given concrete examples about the home group/expert group thingy. And I used almost all of my voice already on the first 45 minutes...

We made a couple of last minute changes to the course on the way to university. First, we decided to change the language to Finnish to get people talking more easily. Second, we decided to get all the groups formed right away. Good decisions, although the talking part did not yet flourish. One reason for that might be my behaviour: I should have asked more and encouraged people to talk.

People for two almost full dojos, nice mixture of students, no-one fell into sleep (saw plenty of yawns though) and I hope no-one got too scared either... The second lecture was about testing. I gave a couple of really bad answers (especially the one for Antero's note about "definition of a good tester") but some of the parts went quite well. Especially the visualization of test levels was something I kinda liked. The idea, I mean. Hopefully it was clear also.

Exciting course ahead. I could already see that there are coders and non-coders participating. You just see it from the reactions and faces. This looks perfect. Let's hope we can get at least one from both sides excited on testing. They will have a bright future ahead.

Dilbert on Test Automation

Trying out the links...
http://dilbert.com/strips/comic/2007-07-28/