Interoperable interactive geometry for Europe
I forgot my login data

Report a bug

Fan club

Quick Intro Videos
click to start movie
Create A Simple
GeoGebra Resource (25Mb)
click to start movie
Filing a review
click to start movie
Find a Resource

This platform is brought to you by the intergeo project, funded under the eContent Plus programme of the European commission and by partners

D6.2: DGS publication best practice


This is the preliminary text of the Deliverable 6.2 which we expect the WP6 team to elaborate. Editing is probably best done with the press of edit.

============== Sophie: I've now problem with my firefox and cannot edit with safary

============== Tomas:

I think this Deliverable: "Report on best practises for DGS content" (as it is officially named) should start with some indications about its context, purpose and approach. What follows is just a suggested text in this direction:

Context: This Deliverable D.6.2 arises as a response to the following considerations in the DoW (description of the WP6 tasks):

➢ In a first round, selected content will be tested in classrooms, and the results will be added to the content database. Also, a report on best practises for the use of DGS content in the classroom will be prepared which shall help the other partners and external testers to improve their work for the next round of testing.After the report on best practises is available, external testers will be actively sought and invited to submit their test results to the database.

Purpose: Therefore, the purpose of this Deliverable is to provide a sound and comprehensive guiding manual for future testers of DGS content within the Intergeo repository. This guiding document stems from the framework for quality described at Deliverable D.6.1, on quality standards.

Approach: To this purpose

--we will start by reflecting on users' needs, since their satisfaction has to be considered as a "piece of conviction" for measuring the sucess/failure of a given resource.

--Moreover, the different kind of DGS resources has to be taken into consideration for an adequate evaluation. Thus, a specific section presenting and classifying the rich variety of DGS publications (even if they are not all equally present in the Intergeo repository) is included in this report.

--Next, we will describe in detail how to proceed in the Intergeo platform in order to perform a sound evaluation of a concrete resource. This is, in some sense, the "core" of this report.

--In this context, the analysis of already performed evaluation seems a useful information for the future testers: lessons we have all learned from the previous (and quite intensive) practice, with circa 2000 reports.

--Previously, a summary of the evolution of the platform regarding the evaluation process (for instance, changes in the initially devised questionnaire) seems also quite useful, as it helps understanding the history of the current Intergeo library of reviews.

--We will end the report with some conclusions and internal suggestions to improve our evaluation mechanism.


Answering needs (Christian)

I have to rewrite all that jazz to a more down to earth version.

The teacher searches the server for resources to teach in her/his classroom; once found, s/he enrolls as a user by providing an a priori assessment and scheduling the course. After an appropriation phase, the course is taught and an a posteriori evaluation is given. S/he can then be promoted to the role of tutor of the resource.

StageDescriptionTeacher needs
Teacher needs and interest in the offer of the projectAt this point, the teacher discovers that our project is offering resources, which may meet his/her needs.Between this stage and the subsequent step in which the teacher will join the project, there is a need to satisfy the information requests from the teacher, to inform him/her about the features of the resources, how to register, the necessary technology equipment, the characteristics of interactive geometry teaching
Teacher joins the projectOnce the teacher is registered, some other needs arise: At this point, it is very important that the project offers appropriate ways to meet these needs. The process of `joining' is intended to organize and control the performance of these aspects.Need to inform the teacher on the use of the technology: virtual learning environments, interactive geometry, beamer technology Need to set-up schedules, rules of work Need to receive administrative and technical support
Teacher is ready to enroll as a user of available resourcesAt this moment the teacher has all the information and skills to start using any resource. During this and the following two stages the teacher may also need technological support, administrative support or other servicesNeed to find and obtain resources from the repository Need for technical support Need for administrative support
Teacher enrolls as a user of a resourceStart of the learning/teaching processNeed to receive learning materials, teaching materials and other resources Need to receive tutoring support Need to receive technical support Need to receive administrative support
Teacher performs the teaching eventEnd of the teaching processNeed to give feedback regarding the teaching experience Need to give suggestions regarding the improvement of the resource* Need to receive a promotion (as a tutor or author) if interested

The teacher using a resource revolves around the needs described in the previous table. But the resource itself follows a similar cyclic process, it has to be planned, designed, produced, prepared, advertised, usage has to be guided and supported, pedagogically and technically, a feedback has to be given during the appropriation stage and after the teaching event.

The author of a resource has several hats on her head to manage this cycle. In nowadays interactive geometry communities, the author is very often confused with the only role of content producer. She indeed provides the contents of the learning resource, but other roles have to be taken into account, she states the intellectual property license; planning and designing, for example, have as well several aspects, whether pedagogical or technical.

It is nevertheless true that due to its technical aspect, interactive geometry needs good learning material producers, experts of their tools and understanding the limit of interoperability. They really are the ``kings in the intergeo castle'' and even though we wish to point out that their role is multiple, the community will continue to summarize it under the term ``author''.

In order for a teacher contemplating the use of a resource to actually use it in the classroom, some motivation and guidance are required. This is the main role of the tutors. The author of a resource is its first tutor but he can be joined in this task by other teachers that used and liked the resource and who want to promote its use by helping others coping with problems they might encounter. This help is mainly done through collaborative tools such as forums and chats embedded in the platform and attached to the resource. The output of this guidance is the adoption by the teacher of a schedule for the use of the resource, both in her teaching progression throughout the year so that it will fit in the course, but as well locally the planning of the resource during the teaching event itself. This is summarized in an individual teaching plan where all the resources from the project used by this particular teacher in a given class are planned, it is a place where the learners are sent in order to retrieve the assigned activities.

The following paragraph is more or less a dream, what should we do with it, forget about it?

Just as a teacher can be promoted to the role of tutor, after a cycle, the resource does not necessarily die and can be improved by taking quality steps towards a revised version. The quality manager of a resource is going to be, at first, in a bottom-up manner, the author of that resource. More elaborate quality management of resources through their integration in subject areas managed in a top-down approach is to be encouraged but the project will not try to organize this more than providing the tools to the community for its self-organization. Before this organization process bootstraps itself, it is the responsibility of the author herself to establish the quality approach, to budget the human costs of quality steps, to establish or not training programs and optionally to take advice from the coordinator of the subject area if there is one in order for the resource to fit in a general learning plan. She will be helped in this by the feedbacks from users.

This feedback consists in assessments from the users through filled questionnaires and in forum interaction. The users who answer to questionnaires will be mainly teachers who use or plan to use the resource, or the author herself, but can be as well didactical experts contacted through the coordinator, who conduct researches on the use of the content from our project. This feedback from the user is quantifiable and statistical when it comes from the numerical values collected by the questionnaire, but consists as well in verbal exchanges, linked to the questionnaire or on the forum attached to the resource, acting as a mailing list. This is especially valuable for the author in order to take the fundamental quality step of revising the resource, whether reusing it in another context or simply producing an incremental version.

It has to be pointed out that according to the license given to the resource by the author, other users may or may not take on their own hand this quality step of revising the resource. The first authoring activity, which is often overlooked or forgot, is to choose a distributing license. The intergeo project promotes the use of open licenses such as the Creative Common Share Alike so that adaptations, whether light such as translations, or heavy such as redesigns of pedagogical goals, can be undertaken by teachers other than the author. It is specially the case of tutors, their natural role is to be promoted from a pedagogical tutor to the author of the next version. Such endorsements exist in open-source software projects where commiters feel responsible for the project. In the same manner, we think that teachers will organize in teams of tutors and that these tutors will promote improvements and will commit new versions. The project will bootstrap this approach on selected contents during the quality testing that is now happening in the third year of the project.

Didactical contract

An implicit contract binds the needs of the author and the needs of the teacher. The later needs resources of good quality and the former needs recognition for the work that was put in creating the resource.

Users are bound by interest, not interest for money, but interest for knowledge. The use of the platform, although provided for free, in the sense that you don't have to pay for it, is not provided without moral obligations. Respect for the work of others is the main pillar of our philosophy. Tokens of respect is the currency that we exchange in the intergeo project. That means that

  • if you upload an educational resource, it should have some interesting facets; half-baked resources are ok if it is explicitly labeled as an idea begging from improvement by others. It means as well to respect fellow teachers opinions on your work; if they invested time reporting on your resource, please consider it in good faith as venues for improvement for your resource. If you don't act on it, or if you disagree with the opinions of users, don't be upset if others modify them whenever the license allows them to do so. The project is here to foster your resources like growing evolving organisms.
  • if you use an educational resource in your classroom, you are expected to report on that use. We are expecting this quality report from users, in order for the project to be useful for everybody; and authors are expecting this feedback from their fellow teachers, as a sign of respect. It is especially the case for groups releasing their work. Please consider that we value your feedback and you should value it as well, not giving it lightly but giving it eagerly.
Respect for good work means as well that quality reviews are not always praises. When reporting, you should assume good faith from the author and should be constructive in your critics but weak points should be pointed out for the author to be able to iron them out.

Work done since last deliverable (Jana & Sophie but others welcome)

Usability of the platform

Tomas: I do not understand precisely the goal of this section in this deliverable.

Perhaps one could just rephrase here some data from the current D4.7

It could be also an occasion to make a (light) comment about the difficulties and delay (as stated in D5.2 or in D4.6 ) in having the platform ready, as a partial explanation for the delay of this D6.2.

Number of resources

Tomas: same as above. But one could use also in this section some data from the Deliverable D5.4 concerning 3rd party content. It has nice statistics.

Evolution of the questionnaire

In this section, we describe the elaboration of the questionnaire according to the agenda presented below:

  Agenda of the questionnaire design an its successive tests
January – March 2008
First version of the questionnaire (reported in D6.1 – Mercat et al., 2008)
July 2008 
Pilot experiment of the first version of the questionnairewith Brazilian teachers in the framework of an in-service teacher training (reported in D6.1, op. cit.)
Objectives: identify what are “good quality resources” for the teachers; validate defined quality criteria and indicators 
July –August 2008
Second version of the questionnaire
September 2008Second experiment testing the second version of the questionnaire with teachers and teachers educators in the framework of a workshop (Iberocabri 2008, Cordoba Argentina)
Objectives: validatethe choices made in the elaboration of the new version of thequestionnaire
December 2008 x January 2009
Beginning of the on line questionnaire use, first reviews recorded
January - April 2009Third experiment testing the questionnaire with 6 French teachers using three resources presenting different characteristics (Master thesis)
Objectives: deeper analysis of the relevance and clarity of thequestions, identify what are “good quality resources” for these teachers
April 2009Third version of the questionnaire (reformulation of theitems)
June 2009Fourth experiment testing the questionnaire with French teachers and teacher educators during a workshop at the Institute of Research in Mathematics Education (IREM)
September 2009Analysis of the early reviews on i2geo website
October2009Fourth version of the questionnaire (new features added to thequestionnaire user interface)
September – November 2009
Beginning of a massive use of the review system

First version of the questionnaire

The relevant features of a resource presented in the §5 of the deliverable D6.1 (Mercat et al., 2008) were used as a basis to elaborate a first version of the user’s questionnaire. The chosen methodology described in the §4.5 of the D6.1, led us to elaborate a list of characteristics or features of a resource related to its mathematical, didactical and pedagogical quality. We wished this list to be as complete as possible addressing all aspects of the resource that we could think of. These characteristics were classified into 5 classes considered as relevant criteria for the resource quality: (1) appropriation of the resource by the teacher, (2) added value of dynamic geometry, (3) learning potential of the resource, (4) technical quality of the resource, and (5) quality of metadata associated to the resource. For each of these criteria, we defined a set of indicators ensuring that a given criterion is met. Each indicator was formulated as a question that could be given an appreciation out of 5 degrees, ranking from “I totally agree” to “I do not agree at all”. Thus, we produced a first questionnaire, not implemented yet, that made possible first pilot experiments. Given the length of the questionnaire, it seemed necessary to start by proposing a lighter version focusing on a few large questions (one per class) addressing globally each aspect of the resource. At the same time, the user will have the possibility to deepen her/his answer by answering more precise questions related to aspects s/he will wish to analyse further, according to her/his expertise. Results of the first experiment carried out in the framework of an in-service teacher training course, reported in the D4.3 (Libbrecht et al., 2008), showed that the teachers adopted the questionnaire as a useful tool for a resource analysis, drawing their attention to the aspects of it that they would have certainly missed without the help of detailed and accurate questions. Thus, the experiment validated the overall structure of the questionnaire and the relevance of the criteria and the indicators we had defined. However, it also showed a need to reformulate some questions that appeared as ambiguous and thus leading to possible different interpretations.

Second and third versions of the questionnaire

While searching for better wording of some questions, we realised that some of the criteria previously defined were addressing different aspects of a resource. For instance, the “appropriation of a resource by the teacher” relied at the same time on the mathematical content of the resource, on the content of the associated dynamic geometry files and on the suggested enactment of the resource with the learners, or the “learning potential of the resource” addressed the issues of the adequacy between pedagogical goals and possible learning through the activities proposed in the resource, the learner’s activity and the analysis of the software in terms of the feedback provided by the dynamic geometry. It would thus have been difficult to clearly identify aspects of the resource to improve since a given criterion mixed up several different aspects. We thus proceeded to another classification of indicators in order to specify better various aspects of a resource to analyse and to evaluate. As a result, we proposed eight classes of indicators corresponding to the following aspects of a resource : (1) metadata, (2) technical aspect, (3) mathematical dimension of the content, (4) instrumental dimension of the content, (5) potentialities of DG, (6) didactical implementation, (7) pedagogical implementation, (8) integration of the resource into a teaching sequence.

=> insert a paragraphe about / rewording of the questionnaire in order to transform questions into assertions, and also always in the sense were if you agree then it is positive for the ressource. Also mentionning the existing "tips" to help the user to better understand the assertion (we still call it "a question").

In what follows, we give a brief rationale for each criterion and the associated indicators. The tables and screenshots correspond to the current version of the questionnaire. Let us recall that the questionnaire can be used for a quick review, i.e. the user can give a general appreciation of a given criterion at a global level (corresponding to the bold questions in the tables 1 - 8), or proceed to a detailed review of aspects s/he wishes to analyse further, according to her/his expertise, by answering more precise questions associated to the indicators defining the quality of the resource according to the given criterion.

Metadata associated to a resource

As stated in the D2.4 (Hendrix et al., 2008), metadata specification aims at describing “the abstract fields that represent the information that users shall browse, catalogue, and query the resources with”. The end-users being mostly secondary school mathematics teachers, the most important metadata allowing browsing and searching for adequate resources are mathematical topic to be taught, school level(s), prerequisites in terms of both mathematical and technical knowledge and skills, pedagogical goals, modalities of use of the resource in the classroom and suggested duration of activities proposed by the resource. This criterion and the associated indicators yield the following items:

Table 1. Metadata criterion and indicators.

The description of the resource is complete (topic, notions and competencies, prerequisites, modalities of use, duration)
  • 1. Mathematical topic is clearly stated.
  •  2. Mathematical prerequisites are clearly stated.
     3. Technical prerequisites are clearly stated.
     4. Trained competencies are clearly stated.
     5. Notions at stake are clearly stated.
     6. A modality of use of the resource is suggested (in a computer lab, in an ordinary classroom with a video projector…)
     7. A duration is suggested.

    Figure 1. Screenshot with the metadata criterion.

    Technical aspect of the resource

    In order for a resource to be usable, one has to be able to access the resource files, to exploit the dynamic geometry files with any piece of software (interoperability) and the files should be bug-free. The following items allow evaluation of the technical quality of a resource:

    Table 2. Technical aspect criterion and indicators.

     The files are technically usable and easy to open.
     1. I can access all files.
     2. I can open dynamic geometry files with my piece of software.
     3. There are no bugs in the files.


    Figure 2. Screenshot with the technical aspect criterion.

    Mathematical dimension of the content of a resource

    There is no doubt that, for a resource to be usable in a school context, its content has to be mathematically correct. Adequacy of the content with the curricula allows the evaluation of the resource utility. Finally, mathematical activities need to be in adequacy with the declared educational goals.

    Table 3. Mathematical content criterion and indicators.

     The content is mathematically sound and usable in a classroom to teach the announced notions and competencies.
  • 1. The mathematical content is valid.
  • 2. The announced notions and competencies comply with the curricula for the stated school level(s).
    3. The proposed mathematical activities are in accordance with the announced notions and competencies.

    Figure 3. Screenshot with the mathematical content criterion.

    Instrumental dimension of the content of a resource

    When the resource includes a DG file, it is necessary to check the coherence between the proposed activity and the geometric figure. In addition, the figure should behave as expected. Particular attention should be paid to the handling of limit cases and of numerical values such as measures of lengths and angles. Indeed, the dynamic diagram should behave according to mathematical theories and didactical objectives. Finally, if special functionalities, such as macro-constructions, are used, a description of their operating mode will make easier the appropriation of the resource by a teacher.

    Table 4. Instrumental content criterion and indicators.

     The interaction with the dynamic figures is valid and in accordance with the intended mathematical activity.
    1. Dynamic geometry figures behave in accordance with the intended mathematical activity.
    2. The figures are robust, i.e., they show no ill effects.
    3. Numerical values (measures of angles, lengths) do not hinder the activity.
    4. Advanced functionalities (use of a keyboard, sliders, macro-constructions) are described.

    Figure 4. Screenshot with the instrumental content criterion.

    Potentialities of dynamic geometry

    Numerous researches on DG put forward its potentialities and their contribution to the learning of geometry (Laborde 2002, Lins 2003, Tapan 2006). This dimension aims first at evaluating how these potentialities are exploited in the resource, and more specifically to what extent DG contributes to improve learning activities comparing to paper and pencil environment. Second, its contribution to the achievement of the educational goals is also analysed. This criterion comprises two aspects: (1) specific features of DG offering an added value to the resource, (2) role and use of drag mode.

    Table 5. Potentialities of dynamic geometry criterion and indicators.

    The proposed mathematical activities take advantage of the dynamicgeometry, they cannot be transposed in paper and pencil as they are.
     1. In this activity, drawings are clear and accurate.
    2. Producing different configurations of the same figure is at stake ofthis activity.
    3. This activity leads the learner to explore,experiment and conjecture.
    4. In this activity, the learner can validate visually his/herconjectures.
    5. In this activity, different representations (graphical, numerical,algebraic) interact.
    6. This activity leads the learner to consider geometric properties ofthe figure rather that numerical or graphical happenstances. 
    7. The activity cannot be transposed in paper and pencil as it is.
    8. Dynamic geometry helps reaching pedagogical goals. 
    9. Dragging is used to illustrate, identify or conjecture invariant geometrical properties or relations between objects.
    10. Dragging is used to bring forward conditions of validity of a theorem.


    Figure 5. Screenshot with the potentialities of dynamic geometry criterion.

    Didactical implementation of the resource

    Trouche (2005) points out that a successful integration of ICT tools requires a specific organization of pupil-computer interactions, which he calls “class orchestration”. The author emphasises the importance of instrumental processes management in relation with learning mathematics. For this reason, we are convinced that a quality resource should provide a kind of assistance related to the class orchestration by means of elements concerning the mathematics learning management with technology, which would help the teacher organize favourable conditions for learning. We propose the following items addressing the issue of didactical implementation of a resource:

    Table 6. Didactic implementation criterion and indicators.

     The description of this activity allows its efficient use in learning of the announced notions and competences.
    The activity is built in a way that the learners engage easily in it. 
     2. Hints are given on ways to make the learners start.
     3. The activity is built in a way to foster learners’ initiatives.
     4. Some reports and learners’ snapshots are provided.
     5. Predictable learners’ strategies, both correct and erroneous, are described.
     6. Remedial actions with respect to predictable dead-end strategies are described.
     7. Actions to help the learners change strategies are proposed.
     8. Feedback provided by the software, which is important for the activity, is discussed.
     9. Feedback provided by the software helps the learners progress in solving the activity.
     10. Advice to determine how and when to synthesize findings is given.
     11. Suggestions on how, when and who validates the learners’ productions are given.
     12. Main characteristics of the activity and the effects of theirmodification on the learners’ strategies and learning outcomes aredescribed.

    Figure 6. Screenshot with the didactic implementation criterion.

    Pedagogical implementation of the resource

    In parallel with a didactical implementation of the resource, we propose a criterion related to its pedagogical implementation, which addresses the issues of the classroom management from the material and temporal points of view.

    Table 7. Pedagogical implementation criterion and indicators.

    The description of the activity suggests a way how to enact it.
     1. Possible material configuration is described (one computer per learner, whole class with video projector…).
     2. A temporal organisation of the activity in different phases isproposed (individual work, group work, whole class discussions…).
    3. A management of whole class discussions and of the conclusion of the activity is suggested. 

    Figure 7. Screenshot with the pedagogical implementation criterion.

    Integration of the resource into a teaching sequence

    We hypothesize that a teacher will more likely choose a resource that fits with his/her usual teaching activities. For this reason, we propose a criterion aiming at determining to what extent a given resource is in line with the user’s planned pedagogical sequence: Is it adapted to his/her students, i.e., do they have necessary prerequisites? Will it be then possible to build upon the knowledge constructed in the activity? Is the activity similar to those he/she proposes usually?

    Table 8. Integration into a teaching sequence criterion and indicators.

      The activity is in line with a planned pedagogical sequence.
     1. The mathematical and technical prerequisites are in accordance with the activity.
     2. The learning outcomes can be reused later.
     3. This activity is in line with the usual classroom activities.

    Figure 8. Screenshot with the integration into a teaching sequence criterion.

    Two experiments were carried out with this version of the questionnaire.

    The first one aimed at validating the choices made in the elaboration of this new version and was implemented during a workshop at the IberoCabri conference in Argentina in September 2008. Teachers and teacher educators from different countries analysed 3 selected resources. The second experiment was set up as a part of a Master thesis (Baudoin, 2009) and it aimed at a deeper analysis of the relevance and clarity of the questions and at identifying what are “good quality resources” for the teachers. Six French teachers took part in the experiment and they analysed three selected resources addressing the same mathematical notion but having different characteristics. Both experiments showed that the teachers and teachers educators share our view of a “quality resource”, the choices of quality criteria and indicators seemed thus validated. Moreover, the questionnaire turned out to function as a tool helping teachers analyse a resource and identify its strengths and weaknesses. However, a few ambiguities still remained that led us to search for simpler formulations of the items. The current version of the questionnaire is the result of an ongoing collaboration of the authors of the present deliverable with seven French secondary school mathematics teachers since September 2008. We need to mention yet another experiment carried out in June 2009 aiming at testing the new version of the questionnaire with French teachers and teacher educators during a workshop at the Institute of Research in Mathematics Education (IREM). The purpose was again to question the quality criteria, first without the questionnaire, in order to make the participants bring forward spontaneously their own criteria. In the discussion with the participants, a new criterion emerged addressing the ergonomic dimension of a resource. Actually, one of the resources the participants had to analyse presented a very interesting activity taking a real benefit from the dynamic geometry software, but the participants were bothered by the presentation of the information, which was messed up and gave the participants had hard time to figure out what was the activity proposed to students, what was set up a priori and what concerned the effective enactment of the activity in the classroom. Taking into account the results of this experiment, we proposed to add the following criterion:

    Ergonomic aspect of the resource

    Previous criteria address mostly the content of the resource. We believe that the way the content is presented is as much important and determines the usability and acceptability of the resource. One the one hand, the user needs to find quickly and easily the necessary information allowing the appropriation of the resource. On the other hand, too much information risks to discourage him/her from using the resource. Finally, to ensure reusability of the resource, the later must be modifiable so that any user can adapt it to his/her specific context.

    Table 9. Ergonomic aspect criterion and indicators.

    The resource is user friendly and adaptable.
    1. The amount of information is satisfactory.
    2. The information is clearly presented.
    3. Elements of the resource can be modified to adapt it to one’s needs.

    Figure 9. Screenshot with the ergonomic aspect criterion.

    Moreover, the last version of the questionnaire presents a rather important change: the appreciation scale consists of 4 degrees instead of 5. The reason for this change is to avoid the undecided users mark the middle degree, which is hard to interpret. We thus want to force the users to incline towards either positive or negative opinion related to the criteria and indicators.

    After some tests performed on teachers by Jana and Sophie...

    Evolution of the questionnaire

    Reformulation of questions

    New features added to the questionnaire user interface

    Tomas: here I recall a few items that could be mentioned:

    --the discussion on the possibility about having anonymous evaluations (yes/not/in some sense...). It was a rich discussion, considering pros and cons. Now, I think, it has been implemented in the last version of the evaluation scheme in such a way that one can tick a box asking for anonymity, but the administrator will anyhow know who has made it. I was one of the proposers of the anonymity box, since, at least in the Spanish culture, no one is going to criticize openly a colleague (in particular if the colleague is well known) if the name of he critic person is going to appear right away in the main page of the evaluation list.

    This issue could be linked, as well, with the fact that we have got a small number of, say, negative evaluations...

    --the issue about adding a box to inform about the "a priori" or "a posteriori" condition on a given evaluation. Now this appears right away, but it was not like this before. We expected the teachers to follow our guidelines, but it turned out that most evaluations have been done "from the living room of the teacher house", without "a posteriori" validation. Yet, I think they are useful...

    --the issue about evaluations not appearing IF there was no answer to a major collection of items (say, no answer in one of the group of subquestions). Some teachers have been evaluating a resource and, since they did it "a priori" they felt some of this big groups of questions did not made sense and they did not answer them. Then, till very recently, the evaluation was NOT displayed (but there was not an indication to its author about it). I think this is corrected now and the whole system is more flexible.

    The reason for not displaying it (as Paul has explained to me once) is that the program attempts always to compute the average of the grades assigned to each one of the major groups of questions. You could leave blank one or more subquestions, but not the whole group… Currently I do not know how this average is computed if one leaves blank a whole group.

    --there is a pending issue, about user profile and weights....assigned to the different groups of questions. I do not know if this is work in progress or not.

    Classification of the DGS publications

    Tomas: frankly, I think this detour to the possible types of DGS publications is a little out of place...(it would have been much better placed at the QA Standards Deliverable). At least it should mention, right at the beginning, that in the Intergeo repository is made, essentially of a few of these types (and a precise mention to the most common types in Intergeo should be done).

    Intergeo welcomes every type of models. In this section we describe the different models we encounter, explain the pros and cons, how they address or not the teachers' needs.

    A part about "too much is too much": if there is too much information then it will deter teachers from using the resource. We should not drown the user under a heavy load of documents, but just enough. So we ought to summarize the teachers needs here into practical terms: the level of education, the time it takes to use it, a brief scenario for use, maybe some feedbacks from the pupils, where are the traps?

    Explain the various existing methods of sharing a DGS activity, qualifying their impact on such aspects as

    • re-usability (in which contexts...)
    • applicability of reviews
    • 1.1.1 Web Locations
    Low guarantees on quality and somewhat unpredictable.

    Most widespread practice thus far.

    Copying for appropriation may be very hard.

    The traces were only links and it gave the wrong impression of yet another repository of links (nothing new as far as the resource model goes).

    Will last as long as the host webserver, so we need to move on to locally hosted resources. We should phase this out by sucking web pages out. Be sure to check the license before that: you are allowed to share a link but maybe not to copy paste a whole page. The license on our website is to be associated with the final resource, not the mere link...

    Guideline for software designers for usability: a publish to I2G button for easy publication of resources. 1st step: just as the publish to web button. 2nd step: fill in as well the meta-data from within the software.

    Plain DGS files

    Maybe harder to "see easily" (but lastly enhanced on platform) but by far the easiest to appropriate.

    Plain DGS files have a hope to be easily translatable.

    Plain recipe

    Translatability is hard.

    Curriki system (Marion)

    There are some model types in i2geo which are not often used.


    One example are the Collections. This model comes in handy if you want to put all resources for a specific lesson together in one place. So if you are planing your lesson, you have the option to organise the resources you decided to use in a collection with the name of the lesson subject. How can you create a Collection? In the "My Profile" section, (which you can access through your “My I2GEO” menu on the left), you will find a tab called Collections. With a click on the button "Add a Collection" you can create a new Collection and after entering a few details about it, you can start adding resources. You add single resources to your new collection by using the "Add" link on the right side at the resource content view. A popup will ask you if you want to add the resource to one of your collections or one of the groups you are a member of. If you click on "My Contributions" you can see a list of your collections at any time.

    Lesson Plans:

    Another way to use i2geo in order to structure your lessons or to share ideas with other users, is the possibility to add a Lesson Plan. The “Add a Resource” option you will find under “Contribute”, offers you the choice to add a lesson plan from a template. You can choose between several template types like the Curriki Standard Lesson Plan, the Nortel LearniT Lesson Plan, the ACE Lesson Plan, the Cloud Institute Lesson Plan and the WebQuest. This model system is probably not commonly used because its use includes a lot of work for teachers, due to the many fields that need to be filled out. It is likely, that this feature is also not widely used because it is not always obvious what content belongs in these forms. Every type of Lesson Plan includes the possibility to upload files and you have to fill out the known and necessary fields "Title" and "Description".

    The Standard Curriki Lesson Plan

    The Standard Curriki template offers the simplest lesson structure, with "Learning Objectives" and "Procedures" as the only required content. There are a lot of other fields but they all are optional. These fields are very informative for other users, because they describe the group size, the educational level and the subjects of the lesson in more detail. Fields like "Learning Objectives" and "Procedures" provide options to format your text using an editing toolbar. Here it is obvious what the required fields are about.

    Nortel LearniT Lesson Plan

    Nortel LearniT provides a project-based, technology-integrated template incorporating experiences that Engage, Explore, Explain, Elaborate, Evaluate, and Extend learning, with the option to include Standards. This template is much more detail oriented. Beside the normal fields, you have to give an "Overview" about the lesson's theme and give information about the "Integration of Technology" as well as the planned "Project-activities" of the students. Furthermore, there are the six E-fields, the following description of these fields is from

    Engage: This is the section where the topic is introduced in an interesting way. It is an opportunity to provide some information but to encourage the students to seek out new information. Often this section will arouse a student's curiosity, violate their current notions of the way things ought to be or incite them to pursue answers to their own questions. It is important not to provide them with everything they need to know, but more the motivation to find out. Each lesson plan has an 'essential question' that is the basis for their inquiry. Normally the section will include a few key questions to help direct some of the research in the Explore section.

    Explore: This is the section where the student investigates the topic more thoroughly. In these lessons the investigation takes the form of Internet research but it could equally include original research and the use of a variety of problem solving techniques. What is important is that the students are given the opportunity to "free wheel" their way through the materials and not be over directed. We all know however, that they will need some direction and the teacher can circulate, asking important questions, listening to their interactions and ensuring that they remain on task.

    Explain: As teachers, we will all have to resist the urge to be the ones that do the explaining at this point. This section provides the student with the opportunity to "explain" or in some way demonstrate their grasp of the concepts after completing their research. Normally this section will contain key questions and/or instruction for requiring the student to provide a summary of what they have learned in the Explore section. This can and should, where possible, take the form of class or group discussion as well.

    Elaborate: In this section of the lesson plan the students are expected to work directly on the given assignment. It is their opportunity to demonstrate their application of new information and to present their findings or conclusions to others. It is a good time for submitting materials for evaluation, doing presentations and completing the project or assignment.

    Evaluate: While it is expected that evaluation will continue throughout the process, this is the section where the teacher evaluates the learning that has occurred. We have provided some tools and suggestions for evaluation but anticipate that teachers will want to adapt these tools for their own use. Students normally submit their work or assignments at this point. It is very important at this stage that the students be encouraged to engage in self-evaluation, group evaluation and develop their own tools to do so.

    Extend: This section contains some suggestions for taking the students beyond the lesson. The purpose is to examine ways in which they can bring their findings to others or apply their understanding to new and unfamiliar circumstances. Normally, this type of activity will grow out of their excitement for what they have accomplished. This section is highly student driven, though teachers may want to gently suggest that the students enter their work in a competition or take their displays to other locations outside of their own school.

    Standards: Standards are currently in the process of being integrated, lesson plan by lesson plan. In this section, the lessons are matched with State, Provincial and/or national standards. It is primarily for the information of the teacher and should provide the information necessary to incorporate the lesson into the local board, district or school curriculum.

    ACE Lesson Plan

    The Alliance for Catholic Education provides a template in which lesson details are formatted in a table showing "Time", "Learning Task", and "Methods or Procedures". Here the special fields are about the "Number of days", which it will probably take to teach the students the specific subject, what is the "Prior Knowledge" they should have and what is the "Lesson's Objective". Furthermore you can write down the "Lesson Assessment", the "Procedures" and "Learning Tasks" in the form of an time table. Aside from the time table, there is no editing toolbar or something else in addition to formatting your text.

    Cloud Institute Lesson Plan

    This template offers the "Educate for Sustainability" lesson framework, including ten required fields. You describe a "Lesson Summary" first, which consist of grade, unit, lesson number and class period. This information helps other educators in finding out if this lesson plan could fit to their needs. In addition to a description of the plan's standards, there is a field to note down "Essential Questions", which should arouse interest in the students. With "Guiding Questions" you try to cover the disciplines of the curriculum. The space for "Materials and Resources" gives you and other educators an overview of what is needed for this lesson and where you can find the materials. Furthermore, the plan features a table which characterizes the "Learning Opportunities", "Activities" and "Procedures" in class time. The field "Environmental Modifications" may be required for students with disabilities, limited English proficiency or bilingual students. In addition to that you have a table for "Assessments" and "Scoring Criteria", as well as a field for the "Glossary" and one in order to describe "Students Work".


    This Curriki template offers the structure for creating a student-directed learning activity that requires students to engage with online resources; it includes an evaluation rubric. First you write a short "Introduction" in order to tell the students the "Guiding Question", which the whole WebQuest is based on. This part could describe a problem based on an actual situation with a link to a video or other materials. The next step is a clear description of the "Task" to the students, if the students should use some software-tools, mention it there. A "Process-field" gives you the option to include single steps or hints, which will help the students to reach the solution. The table "Evaluation" should give the students a chance to see how their progress will be graded and evaluated. Finally, the section Conclusion gives you the opportunity to express the lesson's goal once more.

    Resource from Scratch:

    The third model type of Curriki is to add a resource from scratch. You can choose between a simple wiki resource and a simple html resource, the difference between both is only the type of editing toolbar you use. The simple html resource gives you the option to write html-code into a textfield or add media files. The simple wiki resource editor helps to format the text and it is possible to include tables or images. In both cases there is only one field beside the normal required fields (like Title, Description and Instructional Component Type) where you can write down what you think is necessary. So if you want to write down your lesson plan without fixed required fields, you should use one of these two options.

    An elaborate lesson plan

    Easiest to use. Probably with the biggest need to adapt. Therefore: needs easy appropriation and editability, e.g. if made in i2geo. May drown a teacher in too detailed instructions.

    The SFODEM model

    Multiple-facets of the resources, its usage instructions, its ....

    Best documented but, precisely for this reason, probably the hardest to maintain?

    Interactive Exercises

    Can be very useful and very efficient for learning. Contrary to DGS exercises, which are documents, most of the interactive exercises are actual programmes whose modifications require a developer, and an access to the source.

    The Pairform@nce model for quality evaluation

    Pairform@nce,, is a French national program for in-service teachers education (Soury-Lavergne and al. 2009). It aims to the professional development of teachers regarding technologies. The two main principles in Pairform@nce are:

    • the collaboration among teachers for the design of teaching sequences to be tested in class;
    • the use of an on line platform to support autonomous or collaborative learning as well as in presence and at distance interactions.
    Pairform@nce organises teachers education sessions, which last many weeks, implemented by teachers educators using the platform. The underlining idea is that it is possible to share among the teachers educators the training paths available on the platform. Thus, some teachers educators are asked to design training paths for the purpose of other teachers educators. These training paths gather the necessary resources, propose an organisation of the teachers’ activities in seven steps, support the teacher educator by providing specific tools and means etc… The quality process in Pairform@nce concerns the design of the training path in first place. The resources in Pairform@nce can be identified at two levels: each training path can be considered as a resource and each element associated to a training path is also a resource. Then, the quality evaluation in Pairform@nce addressed the training path as well as the collection of resources attached to it.

    There is a major difference between Pairform@nce and Intergeo which impact their respective quality process. In Pairform@nce, training paths are produced in response to a call, which frames its content and the training paths must be designed according to the “guidelines for the designer”. Whereas in Intergeo, resources are totally freely designed and added on the platform, without having to respect any characteristics (other than being about learning and teaching mathematics with dynamic geometry). The consequence is that, in Pairform@nce, the quality review system is mainly about checking if the designed training path fits the recommendations of the guidelines and in Intergeo, the quality review system is a way to organise and characterise a posteriori the already added resource, with the aim to favor its improvement. Nevertheless, some elements of the quality evaluation in Pairform@nce can be enlightening for Intergeo.

    The quality grid which is used to evaluate training path has 6 dimensions:

    • scientific quality,
    • pedagogical quality (concerning the training paths contents),
    • engineering quality (training scenario, methods and technics used in the activities proposed to the teachers),
    • ergonomic quality,
    • media quality (relevance of the choosen media and way to display information),
    • juridical quality
    The dimensions address either each resource attached to the path or the training path as a whole. The few subquestions (about 3 or 4 and up to 9 for the engineering quality) can be answered by 4 levels of agreement with the extra possibility to express the “I don’t know” answer.

    An other major difference between the two projects is that the quality evaluation is manually treated in Pairform@nce, leading to the acceptation of the training path and its display on a national catalog, whereas it is automatically computed in Intergeo, leading to a stars label attached to the resource. Moreover, the quality expertise is private, anonymous and communicated only to the designers of the training path in Pairform@nce, when reviews are public, just as their authors in Intergeo. And to be complete, in Pairform@nce, quality evaluations are done by experts, in Intergeo our challenge is to make every user able to participate to the quality process and the improvment of  resources.

    Types of resources available on the Intergeo platform

    There are more than 1500 resources on the Intergeo platform by the end of November 2009. Let us first look at the content of these resources. The following types of resources can be found at the Intergeo platform.

    Plain dynamic geometry file resource

    Plain dynamic geometry file resource, containing most of the time an animation aimed at illustrating some geometric property or relationship between geometric objects. The file can be downloadable or it can be run on the web. These resources were usually created by a teacher in a given context. Although they are not directly usable as they are by a learner or in a different context, they can be used by another teacher who wishes to integrate them into a new teaching sequence. Figure 16 shows an example of such resource running on the web, but it can also be downloaded. Educational goals, as well as school level to which the resource can be applied may be specified in the description of the resource.


    Figure 16. Example of plain dynamic geometry resource titled “Rolling circle”.

    Simple resource

    “Simple” resource, i.e., resource consisting of a single dynamic geometry file accompanied by a short text, usually aiming at a student, explaining what to do with the objects in the file. The file can be downloaded or run on the web. The resource “Tvary vznikle z pulkruznice” (“Forms created from a semicircle”), (cf. Figure 17) is of this type. It proposes a construction activity, the instructions for the student being integrated into the DG file. In this category, we find also resources that are web links to pages where a dynamic geometry figure can be run, e. g. the resource “Propriété de l’aire du triangle” (Property of the area of a triangle) containing the link to an activity to be run on the web. Similarly as in the case of a plain DG file, educational goals and other information useful for a teacher may be specified in the description of the resource.


    Figure 17. Resource ““Tvary vznikle z pulkruznice” (“Forms created from a semicircle”)”.

    “Scripted” resource

    “Scripted” resource, i.e., resource that contains a dynamic geometry file together with an accompanying text (Word or PDF files for example) aimed at teachers providing necessary information for the implementation of the resource with learners (e.g., a scenario of use of the resource). These resources are autonomous objects containing everything that is needed to make them operate at both a pedagogical and a technical levels. They aim at reaching a specific educational goal. The resource “Introduction au cosinus” (Introduction to cosine) is an example of this type of resource. It contains a DG file (Figure 18), a student worksheet (Figure 19) and a teacher file (Figure 20) specifying educational goals, school level, duration of the proposed activity, modalities of use of the activity, benefits of using dynamic geometry software to work the activity and providing some comments about the resource.


    Figure 18. Cabri file contained in the resource “Introduction to cosine”.


    Figure 19. Student worksheet contained in the resource “Introduction to cosine”.


    Figure 20. A teacher file contained in the resource “Introduction to cosine"

    "Teaching sequence” resource

    “Teaching sequence” resource, i.e. a resource that proposes several activities aimed at reaching a specific educational goal through a predefined progression of activities. Such a resource can thus contain several DG files and provides necessary information for a teacher to support his/her use of the resource in the classroom. The following resource titled “Définition symétrie axiale” (Defining reflection symmetry) is an example of this type of resource. This resource, running on the web, proposes a series of guided activities for students organized into 8 phases through which the students explore properties of reflection symmetry. A teacher file explains the overall organization of the sequence and provides other information about students’ prerequisites, adequacy of the educational goals with respect to the curriculum, benefits of using DGS etc. Another example is the resource “Areas de poligonos” (Areas of polygons), that points to the web page proposing a sequence of activities aiming at learning formulas to calculate areas of various polygons.

    “Series of activities” resource

    “Series of activities” resource, a resource containing various activities independent from each other. Such a resource can hardly be described by accurate metadata since each activity may aim at a different educational goal, address different aspects of a given mathematical topic. A good example of such a resource is the following one, called “Collection d’activités en relation avec les compétences du chapitre sur les fonctions: fonctions dérivées et fonctions associées. Fichiers interactifs couvrant l’ensemble des items pour tous les niveaux” (A collection of activities related to competences from the chapter about functions: Interactive files covering all items of all levels). The author says in the description that this collection is part of 3000 files addressing various competencies that cannot be all specified. Moreover, a resource of this type can hardly be evaluated by the implemented review system, which would rather apply to each individual activity from the collection.


    Figure 21. Example of “series of activities” resource.

    Other types of resources

    In this category we list and exemplify resources that do not belong to any of the above mentioned categories:

    1. Excerpts of curricula (e.g., “Spanish curricula for courses…”)
    2. Links to various curricula (e.g.,
    3. Photos as pedagogical resources (e.g., “Conics around us”)
    4. Non pedagogical resources, such as Intergeo Translator Manual
    5. etc...

    The choice of a type of a resource will have an impact on the searchability, the appropriation and the subsequent reusability of the resource by the users, as well as on the evaluation process. Advantages, limits and consequences on the evaluation process of the above mentioned types of resources are summarized in the table below:

     AdvantagesLimitsRelevance of the evaluation process
    Type 1
    Plain dynamic geometry file
    Type 2
    Simple resource
      Reusability should be favoured as the resource is not bound to any specific context. The need to imagine possible implementation may foster appropriation. Easy translation.- Using the resource requires making one’s own interpretation on possible implementation.
    - Reusability requires accurate description by metadata.
    * Some of the quality criteria may not be relevant (e.g., didactical and pedagogical implementation)
    Type 3 Scripted resource
    Type 4
    Teaching sequence
    - A possible didactical and/or pedagogical implementation is provided,which can favour appropriation of the resource by a teacher.
    - Clearly identified educational goals in terms of mathematical notions and competencies make the search more efficient.
    - Language barrier
    - Too much congealed scenario risks to make difficult the adaptation of the resource and thus its appropriation.
    - Too much information risks to deter teachers from using the resource. 
    The review system has been conceived for these types of resources. 
    Type 5
    Series of activities
     - It may be difficult to associate to such resources accurate metadata, which will affect their searchability.
    - Usability is affected as well since the user is not helped in findingan appropriate activity among the big number of different activities.
    The review system does not apply to this kind of resources. 
     Table 10. The pros and cons of the different types of resources available at the Intergeo platform.

    Concluding remarks. Intergeo project differs from other projects and learning object repositories by the fact that it welcomes all types of resources. Other resource repositories, such as Pairform@nce for example, require these to be designed according to predefined “guidelines for authors”. Such freedom in the design of pedagogical resources leads to a great variety of different types of resources, which impacts their searchability, appropriation and review process. As is shown in the table 10, the first 4 types of resources present some interesting advantages, while the 5th type can (and should) be avoided by simply extracting and depositing each individual resource at the platform. The types 1 and 2 can be transformed into the types 3 or 4 if the authors provide some information about possible or effective implementation of the resource with learners. Clearly this would help other teachers reuse the resources. In the following section, we develop guidelines for authors aiming at helping them design useful and reusable resources.

    Guidelines and users manual

    Related Guidelines (Everybody, please summarise your knowledge here)

    It would be useful to browse best-practice guidelines I feel and situate our work wrt it.

    Finding the right resource

    You intend to teach in the classroom using educational resources from the intergeo project and you are planning your course, shopping around on the project platform. There are several steps to take in order to successfully find the suitable resource for you.

    What are you looking for?

    You know the educational level of the class you intend to teach. You should first identify the competencies you intend to train in your students by listing keywords that come to your mind. You should be aware of the type of resources that you are prepared to use, should it be ready made with pedagogical advice or do you feel confident enough to make your own interpretation of the way to conduct the teaching? The teaching mode, or ~~instructional type~~ is important, do you prefer an activity in the class room like the exploration of a concept, a homework, an exercise, to use the beamer yourself in the classroom or is it intended for individual use by students in the computers room, or for a self-study phase? The technical issues are as well to be considered, which software are you more confident with, which features of the software are you familiar with?

    Of course, depending on what you will or will not find, some of these parameters can evolve.

    Tomas: the above paragraph is pretty good, but, two comments:

    ---the word "competencies" is introduced here for the first time in this Deliverable and it has specific meanings in the educational context of some countries. I think it could be wise to mention here that we are using it in the precise sense described in the Deliverable XXX (the one that introduced GeoSkills, etc.)

    ---as there are many different types of DGS publications, there are also many more different things a teacher might be looking for when searching in the Intergeo repository. He/She might be looking also for a resource created by a concrete colleague he/she remembers was very interesting (the resource). Or, for a resource with some German annotations, because he/she is teaching at a bilingual institution or because he/she only masters German language...Or he/she might be interested in resources that can be dynamically dragged in a web browser, because he/she does not have any DGS… Some of these needs are not currently addressed in the Intergeo search tool, perhaps we should mention it (and perhaps refer to them later on, in the future work or conclusions section).


    In order to search for a resource, you should go to the intergeo website and simply type your request in the search engine and wait a second for the expansion of the query into competencies and topics to pop-up.

    thalesGeoskills.png The result of the expansion of Théorème de Thalès for a French speaking user in the GeoSkills text box.

    You can search for the text by clicking on the first line of the pop-up or in a much more advanced way, choose the exact competency you want to train with your students.

    thales The result of the search for Théorème de Thalès for a French speaking user.

    In the near future you will be able to reduce this search by adding several competencies, educational levels and preferred language.


    a-resource A very simple resource, a Cabri construction, in Czech, and its annotations within I2Geo for an English speaking user.

    The display of a resource is, thus far, mostly a view of the authors, the topics, the levels, the summary of quality. For some types a more elaborate preview is possible (e.g. the wiki-page types, the picture types). We are currently working on a preview of each construction.

    The set of types of resources is relatively rich, ranging from desktop documents to simple web-links and including on-site-edited videos, as well as wiki-pages.

    An important type is that of resource-collection, which support users' management, for example to indicate the choices made for a particular task.

    Tomas: I find the above two paragraphs a little confusing. After devoting a whole section in this Deliverable to the different types of DGS publications, now the meaning of the expression "the set of types of resources" is not clear to me anymore. I think here it refers to the type of file-formt (video, html, fig, ggb, zip, curriki, etc.), but this should be made clear AND a complete list of current formats and icons should be provided.

    A resource collection

    Analysis of reviews

    Early reviews (Jana)

    The first 55 reviews using the previous version of questionnaire (8 criteria only, the ergonomic aspect of the resource not taken into account yet; 5 degrees of evaluation for each criterion and indicator) have been collected on the Intergeo platform the 24th of september. Among these 55 first reviews, 13 were trials and the other 42 were "serious" reviews. More than a half of them presented either some comments (23 reviews) or detailed answers to a certain amount of indicators (22 reviews). The early reviews were produced by 23 authors, eight of them being members of the consortium and the other 15 seem to be close to the Intergeo project (e.g., teachers collaborating with Intergeo associate partners). These early uses of the review system show that it arouses analysis and involvement from its users. They also enabled us to evaluate to what extent modifications of the review system impact the existing reviews. The high frequency of undecided responses (middle position in the 5-degree scale), in 23 out of 42 reviews, which are difficult to interpret, led us to modify the system, allowing only 4 degrees of agreement, thus forcing a reviewer to chose either a positive or a negative degree.

    quantitative analysis (Marion, Pavel, Roman, SOPHIE, JANA)

    This quantitative analysis concerns either 180 reviewed resources that are accessible and listed the 26th of November or an extract of reviews datas compiled by analytic tools.

    A short overview of 180 reviewed ressources, for the periode of 27-feb-09 to 26-nov-09, gives the following results:108 resources obtain 4 stars, 55 obtain 3 starts, 18 obtain 2 stars and only 3 are evaluated to 1 star.

     Number of reviewed resources 4 stars ****
     3 stars ***
     2 stars **
     1 star *
     180 10654 172
    Table..: Some numbers about the existing reviewed resources (created from February 27th to November 26th 2009).

    These results must be related to existing reviews, because one ressource can be reviewed more than one time. The reviews are extracted from an analytic tool that do not match exactly the list of reviewed ressources obtain on Intergeo platform. 56 resources are reviewed more than one time and some popular ressources are already reviewed up to 9 times! We have recorded 270 different reviews corresponding to 155 different resources. The difference with the 180 accessible reviewed resources on Intergeo platform comes from the fact that currently, some deleted resources still have their reviews registred and still appear while searching for "reviewed resources". They can be identified by the NULL written instead of the name of the resource in the display list.

    45 different reviewers have produced these 270 reviews. Four countries are involved in the reviews: Czech (140 reviews, 52% of the reviews), Spain (55 reviews, 20% of the reviews), France and French speaking country (65 reviews, 24% of the review) and Germany (10 reviews, 4% of the reviews). The 15 reviews in english language have been attributed to the country of the reviewer.

     Total number of reviews (no including empty or test reviews)
     Number of different reviewed resources
     Ressources with more than one review
     Number of different reviewers 45
     Auto-review (by the author of the resource) 27
     Detailled reviews (with at least one sub item answer)
    The review normally produces an overall ranking value from 1 to 5 for the resource. 15 reviews didn't produced a rank for technical problems. The others 255 reviews are distributed in the following way: 164 obtain 5, 66 obtain 4, 18 obtain 3, 5 obtain 2 and 2 obtain 1. These results must not being interpreted by a high level of quality resources gathered on Intergeo but rather as a way users appropriate the review system. This point will be discuss later and must be related to the distribution regarding the country of the reviewer.
    Number of reviews for each value of overallrankingTOTAL per country
     1 2 3 4 5
     135  1
    Germany 10 1   36
     47 2
     63 1 313
    Total per value of overall ranking
     255 2 51866164
    Table: Repartition of 255 reviews according to the country and the value of overallranking.

    Actual review system

    Every registered user is able to review every resource by filling out the Review Class Sheet. This sheet consists of nine question items dealing with different criteria. Every question item is divided into subitems in order to give a more precise review. The value of every item is between one (I do not agree at all) and four (I fully agree). If a user fills out the subitems, the review value of the associated question item is computed as the average of the subitems. Analogously, the resource review value of one user is computed as the average of the question item values. The "Review Summary", which is displayed as the total review value of a resource describes the average of every existing review of this resource. That means the single review values are added and then the result is divided by the count of reviews. You can see the "Review Summary" in the form of four stars on the right side in the resource view. The value is represented by the amount of yellow stars. At the moment it is not necessary to fill out all items in order to create a review, this caused to a lot of incomplete reviews. It seems like many users choose the fast way to create a review and click only on the question items instead of the subitems.

    Tomas: this is true, but a more precise case analysis of this situation is possible. See my comment on this issue, elsewhere in this report.

    It is also possible to create more than one review for the same resource, this should be changed. A lot of reviews are made by the author of the review himself. We do not want to avoid this totally, because it could be that a member of i2geo has found something interesting elsewhere and only wants to share it with other users of i2geo. In this case he is the creator of the resource on i2geo but not really the author. Hence he is the first person who can give an objective review. Therefore we want to change the computation of the review summary in a way, that the resource author on i2geo can create a review but his review should count less than the reviews of other users. In short: we want to create a weighted review system.

    Tomas: it is not clear to me whether this is a desired feature or one currently implemented. Same comment applies to the System for weighted reviews paragraph below. If it is not actually implemented it should not be inserted in this part of the deliverable, but in the Conclusions or Future Work section.

    System for weighted reviews

    Our approach is based on the system Yochai Benker describes in his book "The Wealth of Networks". It is only slightly modified to meet the requirements of i2geo. In this system every user has a karma value between zero and four. A karma value of zero describes a very bad karma while a value of four means the user has a very good karma. Only members of i2geo can create reviews. By restricting the ability to create reviews to the registered users, we are preventing users from giving themselves more than one review by logging out an posting as a guest. Of course we have to add restrictions for members too. It does not make sense to allow users to make more than one review for one resource. They can modify their existing review if they changed their mind about the resource though. A new member, who has not yet written a resource or has not received a review, has a karma value of two. If the average of his own resources' reviews is only one or two his karma value decreases to one or zero. On the other hand the karma value increase to three or four if the average of his reviews is three or four.

    Every review from you on one of your own resources is weighted with one point less of your karma value and with a maximum value of 1. That means, that if you have a bad karma value of one or zero, your reviews are not evaluated if other ones exist. Otherwise if you have a medium or good karma value, your review on own resources is counted fixed by one. Therefore nobody can push his own karma value up to 4 by giving himself good reviews for a long time. A review of a different user is either equally weighted or has a higher weight. Another way to prevent users from manipulating their karma value is to set a fix maximum count of reviews a user can write in a certain period of time. For example 3 reviews per day and 5 reviews in three days.

    Furthermore we should weigh detailed reviews higher than quick reviews. This could be achieved by reducing the weight by one if someone only clicks the major subjects. This way, the review of a user with a high karma value has still a bigger weight as a review from the author or users with bad karma value, but detailed reviews get a higher part of the review summary. This also means, that a user who is creating a quick review on his own resources does not add to the summary value if his karma value is two.

    The review summary of the resource is computed by a weighted average. At the moment we are computing the single review value by a normal average of the chosen items by the user. Then we multiply this value with the karma value of the user, who has made this review. All the results are summed up and then divided by the sum of the users karma values.

    qualitative analysis of the reviews (Jana Sophie)

    The current collection of reviews can be organised into three different types, resulting of three different ways to process the review.

    "Expert" review

    An “expert” review consists in providing answers to almost all indicators and several qualitative comments. Some of such reviews contain comments coming from an effective test of a resource in a classroom setting. Such reviews represent 36% of all reviews and are done mostly by group of teachers who collaborate closely with Intergeo associate partners and present a high involvement in the review process. Clearly, this kind of review should be promoted by Intergeo and some advices to users aiming at helping them produce reviews close to the “expert” level are provided in the Guidelines section of this report.

    Example: (cf. figure 10 below).


    Figure 10: Extract of an "Expert" review about the Planche de Galton resource

    Other examples of expert reviews, in other languages, can be consulted:

    It even exists an expert review in French, by French user about a Spanish resource, written in spanish. It shows that our idea of cross-language tools can be used.

    "Fan" reviews

    A “fan” review consists in positive evaluation of the nine general criteria, without considering the indicators for these. Sometimes a highly positive general comment is provided. It seems that in this case, the user gets a satisfactory general feeling about the resource, s/he likes it and wishes to promote it without providing any information about the reasons why s/he finds the resource interesting or useful. Such reviews may lead to discrepancies between the characteristics of a resource and the evaluation provided, as in the case where a resource that does not suggest any didactical or pedagogical implementation gets positive evaluation related to these criteria. Example of such review is the following one (


    Figure n+1 : "Fan" review about of D. Mentrard site about vectors

    This review is accompanied by the comment “Outstanding and very interesting collection”. The resource under evaluation is a web page proposing a collection of activities related to the mathematical notion of vector (cf. Figure 12). Each activity is a GeoGebra applet having specific educational goals (not specified in the resource), aiming at different mathematical concepts related to the notion of vector (such as vector sum, barycentre, translation…), therefore the collection can be considered as a series of different resources that could be deposited individually on the platform rather than a “single” resource.


    Figure 12. On line resource about Vectors

    The presence of such collections of resources on the platform arises various questions:

    • How such resources can be sought for by the search engine since they cannot be accurately described by the metadata?
    • Even if we suppose that the search engine proposes a collection of resources as a response, looking for usable resources will require going through the different individual resources to be able to choose one of more of them.
    • Moreover, the review system implemented in Intergeo does not apply to collections of resources.
    These considerations led us to reflect on types of resources that facilitate the search for resources and ensure their reusability. Models of resources are discussed in the section (???).

    The following resource consists of a Cabri II file proposing an exercise to students (cf. Figure 13). Moving the sliders associated to the points R N and 0, one gets progressively the analysis of the task (slider R), the solution steps (slider N) and the solution (slider 0).


    Figure 13. Cabri II file proposing an exercise and a solution that the user can progressively display on the screen by using the sliders R, N and 0. Resource named Osovasoumernostnejkratsispojnice on Intergeo

    The only information about the resource provided by the author is its title “Reflection symmetry: the shortest join” and the description “Given a line p and two points A, B in the same half-plane. Construct a point C on the line p such that the sum of the lengths AC+CB is the shortest possible”. There is no information about the educational goal, about possible implementation in a classroom, about the school level. Nevertheless, one of the reviews of the resource gives a maximum satisfaction to each of the nine criteria (cf. Figure 14). One can question how come that almost none of the metadata is specified and the criterion related to the presence of the metadata is highly evaluated. A closer look to the general question suggests an explanation: the question does not ask whether the description of the resource is complete (as it is the case in the French version of the updated questionnaire), but rather whether the user found easily the resource and whether the audience, competencies and themes are adequate. Of course, one can give a satisfactory answer to this question even if the metadata are missing. In fact, the English version of the questionnaire keeps the previous formulations of the general criteria, which do not correspond to the indicators (sub-items). Thus, if a user does not wish to answer detailed questions, but remain at the general level, which is completely legitimate, the evaluation of the resource may present important discrepancies with respect to the content of the resource and thus not reflect the quality of the resource the implemented review system is supposed to reveal. Since the English version has served to translations into the most of European languages, reviews coming from outside France can be biased and there is an urgent need to update the translations of the questionnaire.


    Figure 14: Example of a "fan" review about the resource Osovasoumernostnejkratsispojnice

    Note that “fan” reviews appear in all countries. The preceeding examples are worth to analyse since they point out another issue to be taken into account, the issue of translations of the review system, mainly the questionnaire, into other European languages. 

    “Qualitative general appreciation” review

    This review consists in providing a general qualitative comment on the resource without filling the questionnaire. This kind of review is also useful for a potential user, but it does not allow identifying aspects of the resource that could be improved. Besides, such a review does not count to “stars” computation.



    Figure n+2: An "only commented" review about a Pytagoras theorem ressource

    The comment can be accurate (easy to use and technically valid which correspond to two detailed questions) which is not the case with comments of the fan review type like "trop bien et gigantesque " ("too much and amaizing"). But it do not lead to compute “stars” and the question of the feedback to the reviewer that could lead him/her to detailed its advice is raised.

    Concluding remarks

    The qualitatitve analysis of a few reviews presented above leads us to question the means to support both the authors of resources in a way that produce resources easy to search for, to appropriate and to reuse, and the reviewers of the resources in a way that they contribute not only to ranking resources, but also to the improvement of resources by helping identifying their weak or problematic points. This reflection leads to propose guidelines for both authors and reviewers of resources presented in the last section of this report.

    Some significant results from the analysis per country

    Qualitative as well as a bit quantitative, level of details (shallow or deep, comments) made by author or external users? A bit of sociology of the reviewers if possible (maybe an electronic interview or reaction?) It seems that the reviews are quite different regarding the countries. In France, we made a deep analysis of the question formulations, which has conducted us to reword the questions. The french version of the reveiw class sheet it then slitly different from the ones of other languages. This could explain some difference in the way the review system is used in each country and the obervable resulting reviews.

    France (Jana & Sophie)

    Among the 270  reviews collected by the November 24th, 65 were done by French users or French speaking users, which represents 24% of all reviews. 13 different authors or groups of authors contributed to the review process, nevertheless in a unequal manner, ranging from a unique review up to 13. French contributes for 48 of the 92 detailled and expert reviews.

    Spain (Tomás)

    Among the 270  reviews collected by the 24th of November 2009, 55 were done by Spanish users, which represents 20% of all reviews. They also contribute for 21 of the 98 expert reviews.

    Czech Republic (Pavel & Roman)

    Among the 270  reviews collected by the 24th of November 2009, 140 were done by Czech users, which represents 52% of all reviews. They also contribute for 21 of the 98 expert reviews.

    Germany ?

    Among the 270  reviews collected by the 24th of November 2009, 10 were done by German users, which represents 4% of all reviews. They also contribute for 3 of the 98 expert reviews.

    Conclusion (Jana, Sophie, Tomás)

    Guidelines for reviewers :

    • use the tips and if necessary propose your own tips to explicit the question;
    • complete and modify the review after having tested the resource in class if you have had the opportunity do it;
    • provide element of how you have proceed in the class (adaptation, modification, addition...);
    Guidelines for resources contributors

    • en lien avec les 5 modèles de resources
    Guidelines for developers

    • améliorer le calcul des étoiles, au delà d'une moyenne
    • revoir l'articulation questions générales avec les indicateurs
    • revenir sur la version française pour supprimer les questions redondantes
    • revoir les traductions
    • analyser les tips et les rendre plus pertinents, s'ils correspondent bien (voir l'avis des profs français)

    some other section