Theme 5: Design research

Theme 5: Design research

How can media technology be evaluated?

In the case of the paper ”Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration” a media technology is mainly evaluated through usability evaluation.
According to ISO recommendations, usability is defined “as the effectiveness, efficiency and satisfaction with which specified users can achieve specific goals in particular environments”. The experiments in the paper are therefore considering aspects like: effectiveness, efficiency and satisfaction. A simple indicator of a systems effectiveness is for example the success/fail ratio in completing a certain task. Efficiency is about how much effort is required by the users in order to accomplish a certain task. Satisfaction refers to the comfort and acceptability of the system in relation to the people that are affected by it. These aspects are important to evaluate in order to achieve a better design.

What role will prototypes play in research?
Learning to interact with a certain system should be easy for users so it can be used as effective as possible; therefore prototyping is essential in order to evaluate the system and as a result make iterations of the prototype in order to achieve a better simulation of the intended design.
In order to evaluate an hypothesis in research, especially when the project involves a device which does not exist yet like in the paper, an prototype is useful for testing and evaluating the research.

Why could it be necessary to develop a proof of concept prototype?

Developing a proof of concept prototype (POC) us useful for testing a design idea or assumption, it is helpful if the results of a POC is measurable so that they can affect the decision making process. Since it is a very early stage of the intended product it will provide answers helpful for when evaluating early advantages/disadvantages with the servive/device.

What are characteristics and limitations of prototypes?

The basic idea of prototyping is to understand what requirements of the end service/product. However, the prototype is an early version of the end design and will therefore contain flaws. Testing of the prototype might therefore differ a lot from the end design, which might influence the end design in a negative way. For example considering the case of the paper we read, a test-cellphone were connected to a PCB board which not is a perfect representation of the possible end product, flaws with that prototype might affect participants doing usability tests and might therefore affect the end result.

How can design research be communicated/presented?

If research includes a prototype besides explanatory text about the prototype pictures, concepts, illustrations etc all add value for the reader. In order to evaluate the research, the development process might in some cases be valuable to explain as well. It really depends on if the prototype is an central part of the research how much focus the prototype should have in the paper.

How does a collaborative setting differ from a single user setting as regards methodology used and the results obtained?

A single user setting, there is only one participant at a time, whereas in a collaborative setting there is at least two participants preforming the user testing. The two user settings are different in the sense that the researchers in single user setting have to make the participant to think aloud in order to obtain results, whereas in a collaborative setting participants are more invited to talk about their experiences and thoughts with other participants. The results might vary in the sense where the collaborative setting might result in a possible risk of participants affecting each other, at the same time a collaborative setting might result in a deeper knowledge about the experiences and thoughts than a single user setting would. However, the collaborative setting might be harder to analyze and might not fit all test cases.

How can qualitative and quantitative methods in the same study complement each other?

Using a combination of quantitative and qualitative methods might be a good choice if research includes both quantitative and qualitative aspects. In this case, methods from both areas will complement each other in a good way by using the strengths of each area. Qualitative methods have a tendency to answer the question “why” a phenomenon is as it is, when quantitative methods rather answer the question “if” a phenomenon is as it is. If a research question stated is of both questioning if and why a phenomenon occurs it might be a good idea to mix qualitative and quantitative methods.

How can using both subjective and objective methods give a better understanding of a phenomenon?
This question very much feels like the last one, while subjective methods (qualitative) are used when you want to acquire more personal and in depth information about an individuals thoughts and experiences, objective (quantitative) methods are often used when trying to verify a hypothesis. In that sense, objective methods often verifies if you are correct about your hypothesis while subjective methods examines why and provides more in depth knowledge about the hypothesis and the quantitative answers.