WP5 EO data processing modules implementation

WP5 is intended to complete the development of the RS-IUS component of the proposed EO Data for Habitat Monitoring (EODHaM) system.

A preliminary design of two-stage RS-IUS component of the EODHaM is reported as Figure 2a and Figure 2b in Part B of the proposal. Some Italian sampling sites and the Wales (Western-Europe) test site will be used as training data for the improvement and development of RS-IUS modules based on the following reasons:

  1. Hyperspectral data are also available for some areas.
  2. Different land cover categories characterize the set of the selected sampling areas, ranging from forest land to eroded soil, to wetlands. (See Annex 2)

In order to achieve its objectives, WP5 will consist of the following tasks:

Task 5.1 RS_IUS first stage module improvement

This task is devoted to:

  1. Enhancement of the RS-IUS first-stage SRC module efficiency will be improved by providing source to source code transformation (from the Interactive Data Language [IDL], licensed by ITT Industries, Inc., to the c++ programming language) and enlarge the SRC applicability domain by guaranteeing software maintenance and upgrading, if any.SRC automatically provides as output spectral-based semi-concepts suitable for filling in the well known information gap between concepts in the (3-D) scene and sub-symbolic features in the (2-D) image. A spectral-based semi-concept is a semantic conjecture based solely on the per-pixel (non-contextual) color (spectral, i.e., chromatic and achromatic) properties. For example, if the “color” (spectral signature) of a pixel is, say, green in the visible electromagnetic spectrum, then that pixel is likely to belong to a spectral-based semi-concept called vegetation whose information granularity is, unfortunately, equal or coarser than that of land cover classes (concepts) such as, say, forest or grassland. In other words, a spectral-based semi-concept is equivalent to a (land cover) class set. By definition, different spectral-based semi-concepts are mutually exclusive, i.e., they must feature no spectral overlap. While land cover classes are provided with a superior semantic meaning, but are difficult to be detected automatically, spectral-based semi-concepts are provided with an inferior semantic meaning, but can be detected automatically as shown in recent literature. In operational terms SRC requires neither user-defined parameters nor reference data samples to run, it is accurate, near real-time (e.g., SRC requires less than 5 minutes to map a Landsat image) and robust to changes in the input dataset acquired across time, space and sensors. SRC will provide at any resolution, i.e HR and VHR, from different sensor a preliminary spectral map (primal sketch). SRC maps each input pixel onto a discrete and finite set (56 at the moment of spectral sub-categories (i.e., semi-concepts based on spectral properties exclusively) belonging to six exhaustive and mutually exclusive super-categories (semantic strata) provided with a symbolic meaning as follows: 1) either water or shadow, 2) either snow or ice, 3) clouds, 4) vegetation, 5) either bare soil or built-up, 6) outliers.
  2. Up-scale of RS-IUS of SRC version for mapping hyper-spectral imagery. Partner 11 will cooperate with Partner 12 to software design and fast prototyping of this up-scaled SRC version
  3. Inputs, i.e available EO and ancillary data, derive from Task 4.2 and Task 4.1. Outputs: SRC classifier, at the base of RS-IUS system, is operational. For this reason it will be possible to immediately provide preliminary output spectral maps of Italian and Welsh sampling sites at HR to feed WP6, Task 5.2 for specific class discrimination and Task 5.4 for change detection by comparing same strata at different dates. As an example, burnt forest pixels belonging at date t1 to vegetation stratum will have at data t2, with t1 < t2, , a semantic label of either bare soil or built-up.

The vegetation stratum (and its mask image) will be used in WP6 to immediately localize area to be investigated at finer resolution. As a feedback WP6 will have the opportunity to indicate areas to be monitored at finer resolution. As a result the processing of VHR data will start in the same Task 5.1. Additional outputs are sets of vegetation indices, soil indices to feed WP6.

Task 5.2 RS_IUS second stage modules development

Task 5.2 is devoted to the development of the deductive learning semantic-nets module 1.3 of Figure 2.a (See also Section by identifying and developing the RS-IUS second-stage battery of stratified hierarchical application-specific RS image processing modules to be employed in series with SRC. These modules can be developed independently, which reduces the risk of a major project breakdown. Interpreting complex scenes from EO images often requires, or can benefit from, a knowledge about the scene at hand. This knowledge may provide information about the objects contained in the scene, but also on their spatial arrangement. This type of structural information may be crucial to differentiate between objects that have similar appearance in the images, or to disambiguate confusing situations. Examples can be found in many domains. The key idea in this WP is to derive procedures that are able of translating the spatial arrangements in the knowledge domain (or a model of a scene) as constraints for classification enhancement and semantic labelling. Hence we obtain from a spectral-rule based classification a semantically interpreted scene. In the context of this project, Partner 12 use an AC-n like algorithm [Lecoutre 2009] to maintain arc- or node-consistency between a graph derived from the spectral-rule based classification and the model-derived scene graph. Such an approach is closely related to graph matching methods with the main difference that one can design in the context of CSPs a bunch of specialized algorithms that are tractable. The problem is still however open, and the context of this project presents an opportunity to tackle such issue.These includes:

  • Stratified context-sensitive feature extraction modules. Target contextual features in the (2-D) image domain are listed below.
    • Stratified multi-scale texture features.
    • Segment-based geometric attributes, e.g. area, perimeter, compactness, straightness of boundaries, elongation, rectangularity, number of vertices, etc.
    • Stratified multi-scale morphological attributes (e.g., differential morphological profile).
    • Spatial non-topological relationships between segments, e.g. distance, angle/orientation, etc.
    • Spatial topological relationships between segments, e.g., adjacency, inclusion, etc.
  • To date several context-sensitive image feature extraction modules have been developed, however additional research work is required to guarantee the full operability of this RS-IUS second stage. The extracted features will be the input of the RS_IUS second classification stage
  • Stratified land cover class-specific fuzzy rule-based classification modules (semantic nets). Each hierarchical processing chain consisting of RS image processing software modules must be analyzed, designed, implemented and validated. Their output products can be either single-date LC maps, multi-temporal LCC maps or continuous physical variables (e.g., leaf area index, biomass, etc.).

Outputs of Task 5.2 will include:

  1. LC maps at different spatial resolution to generate habitat map; to feed WP6 for modeling at the habitat level; to feed Task 5.4 for change detection and class transition evaluation.
  2. class /object specific context-sensitive features (textural, morphological, geometric, etc.) useful for habitat characterization and indicator extraction in WP6.
  3. classification modules to feed WP 3.

Task 5.3 Integration of algorithms capable of inductive learning from labelled and unlabeled data and quality evaluation modules

The objective of this WP is twofold: Existing inductive (data-driven) learning modules for data segmentation and classification will be included in the proposed RS-IUS at second stage to incorporate the “stratified” or “layered” approach.

  • Supervised and unsupervised data learning neural networks (distributed systems) developed by Partner 1 in recent years will be included in the RS-IUS second-stage sub-system .
  • Multi-sensor data fusion techniques (to be enforced at the preliminary semantic rather than signal level).
  • Multiple classifier combination modules already developed by Partner 1
  • Algorithms for Hyperspectral image classification developed by Partner 11.

Tools for products quality evaluation will be identified and used to define output product accuracies, uncertainty. Interface modules for input/output will be developed. For each output map, the classification matrix will provide a measure of confidence of the classification quality in terms of overall classification accuracy in percentage (OA) and, for each class, in terms of both user and producer accuracy, k coefficient, but also other techniques already developed by Partner 1 based on the technique of multiple reference cluster maps [Baraldi et al. , 2005]. To face the well-known (but often forgotten) issue of non-injectivity of any quality index, product quality evaluation will be based on the combination of multiple sources of evidence. The description of product quality evaluation procedures will be included in D5.5 of Task5.2.

Outputs will be additional modules for multi-source image classification. The modules will feed WP3 for integration. This will reduce the risk of failure in providing land cover maps to Users. Additional modules for land cover maps quality evaluation will fed WP3 and WP7.

Task 5.4 Change detection modules

This task is devoted to change detection modules improvement and development. Supervised and unsupervised change detection techniques have been already developed by Partner 1 within previous LEWIS-EVG1-CT-2001-00055 project, where change maps where used as intermediate input layer of a landslide early warning system. Change detection techniques will be improved and validated on the base of the ground truth provided by WP4. The availability of an historical time-serie of HR and VHR images and some no-change ground truth can allow the detection of land cover/use changes and the trend of each land cover/use class in time. The information derived by change probability matrices, evaluated on the base of pixel class label frequency, will provide specific probability class-transitions and then the trend of each class.

Inputs to this task will be the maps from Task 5.1, Task 5.2 or Task 5.3 as well as data and info from WP2 and WP4.