Fundamentally, should there be tool tests? Some seem to think not but that would leave us relying on adverts and hearsay. Neither of these offer sufficient or wholly reliable help to a prospective tool buyer.
Who should test tools - magazines or an independent service? Both, perhaps; but magazines need to be much better at testing than they are presently. As one poster noted, there is much evidence of the manufacturer’s puffery coming through. Of course, an attitude of caveat emptor will usually see through the puff to any substance of interest. However, such “tests” are really just rehashed adverts. (But adverts are useful too).
FWW has a record of attempting unbiased and objective tool tests (ie with measured parameters); but the nature of a magazine article is such that there is never enough depth to these tests.
Independent testers need to demonstrate their independence but this begs the question, where do they get their funds? Their time may be freely given, should they have it spare; but we could hardly expect them to buy all the tools tested. Would you pay if it were a subscription service, risking that 90% of the tested tools were of no interest to you? If not, we have to allow manufacturers to give their goods to recognised testers; and even to fund them (but via an independent bursar perhaps).
************
There are two well known test sites for cameras on the web. These offer good models for independent in-depth testing of other machine-types. They give a very detailed description of the subject models then offer results from investigations of several aspects of that model. The same basic parameters are measured and reported on for each and every similar model, giving something of a level playing field.
Some measurements of the cameras are objective (what is often referred to as “scientific”). Specific functional performance (such as the resolving ability in lines per inch) are defined then measured with an instrument.
Other aspects of camera tests are more subjective, covering aspects such as the ergonomics.
Sometimes there is a half way house, where specific features of the camera are measured objectively and then the reviewer opines whether this represents a good, bad or significant design feature. An example is the internal processing software features of a camera, the performance of which can be measured objectively and then discussed as “useful”, “well implemented”, etcetera (or not).
These camera reviews are quite exhaustive, with many HTML sections and pages of description, measurements and opinion. They generally end with a narrative conclusion, a list of good and bad points, with an overall “recommendation level” ranging from “avoid”, through “acceptable”, “recommended” and so forth, up to “excellent”. Example pictures taken at various settings (the camera-tool’s products) are also published, along with comparisons against pictures from a similar camera or against a reference model’s pictures.
***********
If there were a WW tools version of such a test service, what should be the testing parameters? What is the WW analogue of the camera site? Although WW tools are very different from cameras, some common parameters can be identified:
* Description, including type, purpose, maker, cost, availability, etc..
* Engineering quality, including materials used, tolerances, correct functioning, etc.
* Design quality, including ergonomics, build and finish, overall performance, safety provisions, setup requirements, etc.
* Performance, including standard (objective, measurable) tests for that tool type; comparison against results from a reference tool or similar tool(s) in the market; subjective impressions, including those from long term use by 1 or more “associate testers”.
* Failure test, including work-feed overload, possible misuse (eg with wrong material or blade).
* Reliability, via simulated usage cycles and/or long term user reports.
* Summary table of good/bad points
* Recommendation marking, perhaps including marks-out-of-10 for all the above test criteria.
***************
One of the camera test sites has an extensive facility that includes a number of enhancements to the basic test reports.
* Users of cameras can write their own test report, based on their experience.
* There are associated forums dedicated to particular manufacturer brands and, within them, threads that are dedicated to particular camera models and even to specific functions of particular camera models.
* There are links to other sites run by “interested parties”, from the manufacturers, to professional photographers using certain cameras, to individual amateur-enthusiasts and their work.
These extra facilities provide a very rich, albeit less organised, additional information source about any modern camera you care to name. Some contributors are extremely knowledgeable and have published detailed investigations, with much evidence in support, concerning specific aspects of this or that camera.
**************
Is there any reason (other than the funding issue) why FWW could not sponsor a tool review site organised in this more effective fashion? As we know, Knots users would be more than willing to enhance any standard test reports form FWW staff with their extensive user knowledge, especially if their contributions were seen to be made accessible and relate-able to one another.
Finally, perhaps one other improvement to this general model might be considered – the reorganisation and indexing of the good information to found in Knots members’ postings (and the expurgation of much of the dross) into a Wikipedia-style knowledge base. Some have already proposed such a permanent thread for plane making, I believe.
People looking for hard information about tools would then turn to this facility rather than having to discover and wade through tons of stuff (including natterjack noise) via the Knots search engine applied to all the historical postings.