GUIDELINES: SOFTWARE Felix Portmann

Working on time series analysis leads irrevocably to the use of software. Only in special cases manual methods replace or assist software.

Users should take into account that the software use for statistical analysis is threefold:

• Data preparation:

In the beginning, the data exist often in various forms and must be transformed into the format that the specific software can read. Perhaps missing values have to be excluded, marked or replaced by estimated ones. Instantaneous values have to be aggregated to daily, monthly, annual series and annual maximum series or partial duration series have to be generated. Either the statistical software helps with the data preparation or the user has to do it on her/his own.

• Statistical analysis:

At this core analysis, the data is analysed with a statistical software and results in primary form are produced. These may be in the form of tables, graphs, semantic information like rejection of hypothesis, etc. Interpretation of the results follows.

• Presentation of results:

Finally, the primary and conclusive results have to be compiled in a desired form, e.g.

a report, presentation slides, a scientific publication, etc., and not only as plain hardcopy text or graphs on a sheet of paper. The compilation often needs additional treatment, e.g. annotation of graphs, inclusion of additional material, etc., normally done nowadays with software, too, e.g. Office software and Drawing software.

When choosing software, users should be aware of the scope of their analysis, e.g.:

• data checking for calculation of frequency statistics

• number of series to analyse

• form of presentation

For in-house data checking, especially for a small number of series, a manually organised sequence of analyses with simple output might be sufficient. When using large number of series, automatic procedures should be present. Furthermore, when the results have to be included into a final document or presented at a conference, for their inclusion with the specific editing software an accepted format (format of graphical metafile, e.g. WMF, JPG, TIF, PCX GIF, PICT, or of text, e.g. ASCII or RTF) and necessary type and quality (black &

white vs. colour, colour depth, adequate and correct representation of style of text and points, lines, legends, annotations in graphics, pixel resolution, etc) have to be chosen, as retyping is not always possible or feasible. Any format of results supporting the final treatment is more appropriate than a format ensuing additional steps or even impeding the production of the final product in the case of incompatibility. Also, the reproduction form (paper vs. transparent media) and costs (black & white vs. colour, PostScript or not) should be considered. In this respect, hardware for the production of the output may be important to check, e.g. PostScript capability of printer or colour printer. Perhaps adjustments of colours in graphs have to be made to get a readable and ready-to-print or ready-to-photocopy hardcopy.

The specific circumstances can be very different, as the goals of the analysis, the organisational framework and the available hardware may be very different, but decisive, too.

Users should keep in mind that any software may present errors, either open or hidden ones. Therefore, results of statistical software should not be taken for self-evident. Common sense and checking important framework conditions, e.g. sums, can avoid relying on wrong results.

Another problem with software is inappropriate or bad algorithms. E.g., often the so- called “random generators” do not generate truly random series, but biased ones. Besides that, when doing own programming, any correction for sample size should be done in the proper way.

Considering software specifications, the workshop experts concluded that currently no software exists that covers all recommended features; many approaches like permutation or bootstrapping have to be programmed or implemented at your own. Especially standardised time series analysis software often do not accept missing values. Estimation of those often remains a problem that has to be resolved externally from the statistical software.

Therefore, the experts emphasise the need of

• Development of a software with the desired features

The realisation of this goal is beyond the scope of this workshop itself, as the programming effort will be considerable. WMO is informed through its participant of this central view of the workshop experts. Funding of such an enterprise could be proposed e.g. within the 5 Research & Development Framework Programme of the European Commission.

However:

• For specific applications solutions can be programmed at your own either directly, via programming libraries, or within statistical programming frameworks, e.g.

SPLUS. Users should be aware that this takes some time but can result in optimised, very quick analyses within a common data framework.

• Some work can be done by already existing special software or even general purpose statistical software, e.g. SYSTAT. A selected software often presents only partial solutions. When using different software at a time, the raw data sets have to be brought into the specific data format. This requires some efforts, especially pre-processing of missing values that many programmes do not accept. Own programming may be required or very useful for the pre-processing.

• WMO has a collection of programmes within the HOMS (Hydrological Operational Multipurpose System), of which many, but not all, are freely available. A list of freely accessible software for specific tasks, some of them within HOMS, is given in the following table. Please note, that the list is not exhaustive, but has been selected especially to provide information on freely or easily obtainable software which is giving high-quality output and does not normally need very sophisticated hardware.

Experts designed and used these programmes, they can recommend these for specific tasks. For more details, readers may contact either the WMO internet site (http://www.wmo.ch/web/homs/homshome.html), the authors of the software or the author of this contribution.

Key name Descriptive

Specific test included Input data Output Miss.

Values Operation

Key name Descriptive

Specific test included Input data Output Miss.

Values Operation TREND1 Linear trends Ruhr-Univ.

Bochum

Appendix 2 HYDROSPECT -

SOFTWARE FOR DETECTING CHANGES