Torstein Rekkedal
Courses on the WWW - Student Experiences and Attitudes Towards WWW Courses

An Evaluation Report Written for the Leonardo Online Training Project
http://www.nki.no/eeileo/

Introduction

This report is the second evaluation report written for the Leonardo Project, « Multi Media World Wide Web Kernel for Distance Education». The project partners are Ericsson Systems Expertise, Ireland; University of Rome III, Italy; and NKI Distance Education, Norway.

The project will incrementally develop a WWW kernel for distance education, a series of courses to be distributed and taught on the WWW and evaluate these courses with the aim both of developing the kernel itself, developing general didactics for teaching on the WWW and increasing the quality of the individual courses.

This evaluation is based on a questionnaire distributed to all students at NKI Distance Education who completed at least one WWW course during 1997. A pilot evaluation report which covered active students on Internet courses in summer 1997 was presented by Paulsen and Rekkedal (1997) (http://www.nki.no/ekko/for_alle/fagartikler/evaluation/evaluati.htm).

NKI and Teaching on the Internet

NKI is one of the largest non-governmental educational institutions in Norway. NKI comprises The Polytechnic College, NKI Distance Education, the Business Training Centre (NA) and The NKI Publishing House. A further presentation of NKI is available at URL: http://www.nki.no
NKI Distance Education offers more than 60 distance education programmes and more than 300 courses, mainly in technical and vocational subjects. It has about 15,000 students. An extensive network of local partners around the country is established to offer face-to-face classes to support the distance students. The distance education tutors are part-time employees recruited from business, industry, schools and colleges around the country.
The College of Computer Science (now incorporated in the Polytechnic College) was established in 1983. In addition to its ordinary two- and three-year full time programs, it offers a two-year, part-time program in information processing, emphasising programming and system analysis. This part-time program is offered in three modes: face-to-face classes, correspondence courses, and the Electronic College.

The Electronic College

The Electronic College is the metaphor for the integrated offer of NKI distance courses based on course distribution and communication on the Internet. It is designed so that distance students can do all their communication online. In contrast, many of the internationally renowned implementations of `electronic colleges' are actually using computer conferencing as a supplement to correspondence courses or on-campus teaching. The philosophy of the NKI Electronic College has always been to offer programs that are independent of time and space through a system that facilitates flexible communication for administrative, social, and teaching purposes.

Since the first courses were delivered in 1987, one may distinguish between three technological generations of the NKI Electronic College. The first generation system, which was based on the `EKKO computer conferencing' system developed at NKI, was in operation from 1987 to 1993. The second generation, which was Internet based, started in January 1994. The first and second generation experiences are documented in an OECD report (Paulsen and Rekkedal 1996) and in several articles available at http://www.nki.no/ekko/for_alle/fagartikler/.

The third generation comprises web-based courses introduced in 1996.

The Focus of this Evaluation

It was a major intention that the evaluation of the first run of courses based on the WWW kernel developed in co-operation as part of the project should include a majority of questions common for the three partners. Thus, some of the questions could perhaps seem to be more relevant for one partner than for another. A few of the questions were adapted from a previous survey, `Evaluation virtuelle Seminare' (Fritsch 1997). In general we wished to cover questions within three areas:

· the students' attitudes towards WWW-based learning
· their situation concerning use of computers and computer technology
· characteristics of students - who are attracted to WWW courses (1997)
· specific viewpoints on learning on the WWW and the actual courses

Methodology

Survey Method

We have, over the years, experienced that it becomes more and more difficult to receive acceptable response rates to postal questionnaire surveys. We chose to construct a questionnaire including mainly fixed answers to tick.

The questionnaire was distributed to 56 students end of January 1998. We selected all students who had completed one or more courses presenting the course material on the WWW during 1997. An introductory letter explaining the aims of the survey and a free postage return envelope was enclosed with the questionnaire. The answers were returned anonymously, and we sent out a reminder to everyone after approximately 2 weeks.

Response Rate and Representativeness

After 4 weeks we had received 34 completed questionnaires, i. e. a response rate of 61 percent. As the presentation of results will show, the students' attitudes towards learning efficiency of WWW based courses are generally quite positive. However, it could always be questioned whether the respondents constitute a representative sample of the population. To examine whether there were systematic differences between early and late responders, we tested for differences between the first and second half of the respondents on the attitude questions. The reminder was sent after having received approximately half of the completed questionnaires. We tested the differences in answers between the 17 first respondents, as one group, and the 17 last respondents as another using the Wilcoxon-Mann-Whitney test.
The tests showed that there were statistical significant differences between the two groups (probability values .005**, .008**, .031*, .101, .007** and .007**). This means that the early respondents were more favourable towards their experiences from learning in a WWW based course than were the late respondents. From this one should take into account the possibility that the non-answerers perhaps might be less positive than the average of respondents.

Results

The courses

Table 1 gives an overview of the courses and the number of respondents having completed each course. All courses involve the following aspects:
· Learning material presented and distributed on the WWW
· E-mail communication with tutor and fellow students
· Conferencing through a Listserv distributed mailing list
· Assignments for submission corrected and commented on by the tutor.

The courses generally also include ordinary textbooks. The students can enrol in the courses at any time and follow their individual schedule of study progression. This individual flexibility is, according to previous surveys among students (e. g. Rekkedal & Paulsen 1997), seen as a great advantage by the students. On the other hand, this flexibility reduces the possibility of meaningful student-to-student interaction.

Table 1. Programmes and courses offered during the survey period.

24 of the respondents have completed one or more WWW courses included in the Information Network Programme. These students may either have enrolled for the whole programme or just one or more single courses. 5 have completed one or both courses of the Information Technology programme, while 1 respondent has completed the single course in Java-programming and 5 of the respondents have completed the Tutor in distance education course. Except for the last mentioned course, all courses are related to information technology and WWW knowledge and competence.

Attitudes towards learning at a distance in courses taught on the WWW

One important aim of this research was to examine whether offering the courses on the WWW facilitated enrolment, whether the WWW was seen as an efficient way of learning and whether their general experiences had been positive, measured by possible interest for enrolling in another course themselves or if they would consider recommending others to take courses offered on the WWW. To approach these problems from different angles we asked six different questions measured by a 5-graded `Likert scale'. The questions are presented both in English and Norwegian below. The results are presented as percentages of respondents who have ticked the alternatives and the median(Md) of the answers, where the extreme positive end of the scale counts 5 and the negative end counts 1 (Note that in Table 4 were the question is put in the negative form the values of the scale are changed.)

Learning effects

Table 2. From my participation in this course I can state that it is possible to achieve excellence in courses that are taught on the WWW (På bakgrunn av min deltakelse i kurset/kursene mener jeg at det er mulig å få fullt læringsutbytte gjennom kurs som underviser på WWW).

Md=1.7

Courses can be presented in many ways, such as full-time or part-time classes, distance education based on printed material and other types of pre-produced material, correspondence education or e-mail communication with or without conferencing facilities. We have e. g. previously showed that CMC students have achieved better at examinations than both part-time students taught in classes and correspondence students (see Paulsen 1992).

We can clearly conclude that the students in this survey agree and agree strongly that efficient learning can result from distance courses based on WWW. Only 3 students disagreed, none ticked `strongly disagree'.

Table 3. My enrolment in this course was facilitated because it was on the WWW (Det at kurset ble tilbudt over Internett/WWW gjorde det enklere for meg å begynne på kurset).

Md=1.4

It is difficult to say what form of learning the students actually are comparing with. In a previous NKI survey among Internet students (Rekkedal & Paulsen 1997) we asked the students about the probability of enrolling in full time, part time or correspondence courses. It was clear that few of the students would consider taking part time or full time studies. The probability of enrolling in correspondence study (having similar flexibility as WWW courses) was higher (41% on the positive side of the scale compared to 23% for part time study and only 3% for full time).

Thus it is not surprising that more than 80 percent of the respondents confirm that enrolment was facilitated because the course was offered on the Internet/WWW.

Table 4. My learning from the course was damaged because it was on the WWW (Min læring i kurset ble dårligere fordi kurset ble undervist over Internett/WWW).

Md=2.1

As this question was given in a negative form, the value assigned to the alternatives when calculating the statistical measures is opposite to the other questions. Very few agree with the statement that learning was damaged because the course was presented and taught on the WWW. On the other hand, some few students actually indicate that teaching the course on the WWW has influenced their learning negatively.

Table 5. My course results will be just as good as if I had studied face-to-face or in an instructor led training course ( Mine læringsresultater i kurset var like gode som i et ordinært kurs med lærerledet undervisning i klasse).

Md=2.6

For obvious reasons the students have difficulties in comparing learning results from courses on the WWW with possible results from learning by other forms or methods of study. This is probably due to the fact that they have studied this (these) specific course(s) only as distance learning based on the WWW. Still, we may note that there are more students who agree with the statement than disagree. Nearly half of the students are uncertain.

Table 6. I would enrol again in a WWW based course (if there were courses available suited to my learning needs) (Erfaringene mine tilsier at jeg gjerne kunne ta et WWW-kurs ved en senere anledning).

Md=1.4

Concerning the possibility of enrolling again in a WWW based course the majority are very positive. Only one student disagrees with the statement. There should be little doubt that this feedback from the students must be understood as very favourable towards distance learning based on the WWW.

This fact is largely supported also by the even more positive responses to the statement of encouraging friends or colleagues to enrol in a WWW based course. 84 percent of the respondents agrees with the statement (Table 7).

Table 7. I would encourage a colleague or friend to enrol in a WWW based course (Jeg anbefaler gjerne en kollega eller venn å ta kurs basert på undervisning over WWW).

Md=1.2

It is very encouraging that a great majority of the students agree to the statement of encouraging a colleague or friend to enrol in a distance learning course presented and taught on the WWW.

Access to computers and technology and use of computers
Part two of the questionnaire surveyed the students' access to computers and the Internet, their actual use of computers measured by `hours spent at a computer screen' and `computer literacy' (here measured only by knowledge of HTML). These questions, together with a question on who is paying for the course, are supposed to give some information about the market for distance courses on the WWW at present (or more correctly in 1997).

Table 8. How much time did you spend last week looking at a computer screen
(including work, study, leisure)? (Hvor mye tid brukte du ved en dataskjerm siste uke (inkludert arbeid, studier, fritid)?)

Md=13.3

As mentioned, the questionnaire was distributed to students who had completed at least one WWW course. This means that some of the students, taking more than one course, might still be studying with NKI Distance Education while others might not. Without actually having much comparable data so far, we at least would conclude that these students, on an average, spend a considerable amount of hours in front of a computer screen every week, the median time being 13.3 hours. However, in the above mentioned German study (Fritsch 1997) the students spent even more time before the computer screen.

What type of computer did you use for the course?
Nearly all the students (29 (85%) of 34 answers to the question) use a Pentium computer, three students use a 486 computer, one student uses a Unix computer and one uses a Macintosh computer. No student was using a less than 486 computer. Consequently, it seems that concerning capacity and speed we may assume that most students in our WWW courses are using Pentium computers.

Table 9. What was your average waiting time from starting up your computer to getting to the course pages?

NKI Distance Education has chosen to construct course pages in the WWW courses with text, graphics and presentation forms to minimise the amount of data to be transferred, to reduce waiting time as much as possible. The alternative answers were given as subjective judgements and not as objective measures in minutes and seconds. Not surprisingly, as the questionnaire was distributed to course completers only, the last alternative was not ticked by anyone. If these answers are representative for waiting time in general for the web pages in the courses, we would conclude that, as the courses are constructed, the capacity of communication lines are satisfactory for the majority of the students. It seems that our strategy of minimising the load of information in the web pages has been successful so far.

Access to Internet and e-mail before course start
NKI Distance Education started its first pilot courses based on computer conferencing and the internally constructed conferencing system, EKKO, in 1987. Since 1994 we started using the Internet for e-mail and Listserv for group distribution and student interaction. Since the early pilot courses, it has been one important element in the total course package to offer user access and e-mail address at the NKI server. During the last few years we have seen that more and more new enrolees have Internet access and e-mail addresses before they start their courses. Among the respondents in this survey there are 26 (79%) of 33 students who answered that they had an e-mail address before they started their study. Thus, as NKI does not look upon Internet access provision as a primary service, it is time to discuss whether this should be offered in the years to come.

Can you construct or change a page in HTML?
As one measure of `computer literacy' we included in the common questionnaire whether the students were able to construct or change a page in HTML. It is not surprising that the majority of respondents in the NKI sample confirm that they can use HTML - as they have completed courses related to the Internet, WWW, C++ and Java programming languages. It is only the course for `Tutors in Distance Education' that has no direct relation to computer science or the application of Internet tools. 26 of the respondents (76%) answer that they can construct or change a page in HTML.

Table 10. Who is paying for the time and resources you use for the course?
(Hvem betaler for kurset - inkludert tidsbruken?)

The alternative 3 was included in the questionnaire to make it similar to the questionnaires used by the other partners in the Leonardo project. Who is paying for the course and whether in-company students are allowed to spend their paid working hours for study, would perhaps be better covered by two separate questions, as it seems clear that for most respondents the alternative number 5 means the course is paid for by their employer, but that they at least spend some of their leisure time for study. It seems that more than 50 percent of the respondents have their courses paid by their employer. The figures are comparable to previous results from NKI surveys (both among Internet students (Rekkedal & Paulsen 1997) and among Technical College students (Rekkedal 1997) with a tendency of a larger proportion of Internet/WWW courses paid by the employer than other types traditional correspondence study based distance education courses).

Student characteristics
Table 9. Age.

Md=34.3

There are only small differences between these students and distance students at NKI and in general. For instance in a survey among distance students at the NKI Technical college in 1994 (unpublished internal evaluation paper) and in 1997 (Rekkedal 1997) the median age was 32 and 31 years respectively. There is no sign that the NKI WWW courses recruit especially young persons as could perhaps be expected.

Gender
Out of 33 respondents answering the question on gender, 17 (52%) are woman and 16 (49%) are men. The WWW courses enrol both men and women and both gender complete courses. Distance learning through WWW based courses are chosen by and completed by both men and women.

Table 10. Occupation.

`Unemployed' was also given as an alternative, but not ticked by any respondent. There are quite a number of students in the sample. These are probably full time students who take one or more distance course in addition to their ordinary study load, or take a distance course as a personal choice among other subjects credited in their study programme.

Table 11. Previous level of education.

For obvious reasons, most of the students have completed secondary school with university admission certification and the majority has also completed some years of college or university. E.g. most of the students taking the `Information Technology Programme for Teachers' have completed 4 or more years of higher education. `The Information Network Programme' is also a typical further education study for people having completed a college certificate or degree. In principle, students may take single courses involved in college programmes without having earned entrance qualifications for higher education programmes.

Table 12. Reasons for enrolment in the course (one tick only for the most important reason).

It is clearly demonstrated that the reasons for enrolment are mainly job related. The alternative chosen by most respondents is that `they need the competence in their present job'. `To increase general competence' also seems to be important. Some students tick `interest in the subject' as the prime reason for enrolling. We have discussed whether studying on the Internet/WWW, as something new in itself, could stimulate enrolment to distance studies offered on the WWW. This does not seem to be the case. In our previous survey (Rekkedal & Paulsen 1997) no one ticked the alternative `interest in this form of study'. In this survey this alternative is ticked by two respondents. Both had completed the `Tutor in Distance Education' course. Thus, their interest in teaching on the Internet/WWW is not surprising.

Some other viewpoints on the experience of learning on the WWW and the courses
WWW Material in Printed Form
The NKI WWW courses use the web in different ways. In some courses, all the material is distributed on the WWW, in some courses the `Study guide' on the WWW is supplemented by printed text books. In the `Tutor in Distance Education' course the students receive the material also in print, except for the hypertexts with reference to internal and external material. As part of our didactic discussions, whether to supply material also in print and which material to distribute in which form, have to some extent been controversial. Thus, at different occasions we have raised these questions to students. In this survey we simply asked the question: `Would you consider it an advantage if the course material on the WWW was also distributed in printed form?' 26 (79%) answered `yes', while only 7 (21%) of the respondents who answered the question said `no'. Consequently, it seems that the students in general are in favour of receiving the WWW material also in print. Of course, both for technical reasons (some material on the WWW is difficult or impossible to present in print) and for cost reasons, it might not be feasible to supply additional printed material.

Final open-ended questions
This first of the three open questions completing the questionnaire was answered by 29 persons, the second was answered by 25, and the last question (other comments) was answered by 21 persons. The answers had very in different lengths and, of course, many of the answers contained viewpoints on different aspects, which made categorisation quite difficult. As the final comments generally covered viewpoints similar to the two first questions, we have presented these comments as either `appreciated elements' or `elements missing or under-exploited'.

Which elements of the WWW course have you most appreciated?
The answers below cover general advantages of individual distance learning, advantages of communicating electronically, to specific advantages connected with presenting courses on the WWW, viewpoints concerned with the specific course(s) they have studied, and general positive comments on experiences of learning in the WWW based course(s).

General viewpoints on distance learning and flexibility:
- Freedom to study when I wish.
- Freedom to choose time, place and progression.
- Independence of time.
- The possibility of communicating with tutors and fellow students via e-mail, and taking the courses without being locked to certain hours.
- I study at the University of Oslo, so the main importance was the possibility of taking the course without having to be physically present; quick communication between student and tutor.
- Time saving, can study when I wish, very quick feedback from the tutor.
- The flexibility; quick feedback from the tutor; updated learning materials.

Exercises and assignments (aspect of individual distance learning):
- Exercises and assignments.
- The assignments for submission.
- the exercises and assignments on the WWW gave very good insights - and learning through self activity and work.
- Practical assignments.
- Project work (`Information technology Programme for Teachers').

Viewpoints concerned with computer mediated communication:
- Use of e-mail; easier than visiting the post office.
- The use of information technology and use of e-mail.
- It is clear that Internet, e-mail, PC's etc save time; however with these tools documents may disappear and technical difficulties may arise; the threshold for frustration may be low, apart from this the use of information technology in distance teaching is a great advantage.

Viewpoints on the quality of tutor work and feedback:
- High quality and quick tutor responses.
- It is very motivating that the communication with the tutor is so fast.
- Excellent teacher in `WWW presentations'!
- The contact with the tutors - specifically in the course `Network Administration'.

Viewpoints concerning course material on the WWW:
- That everything was on the screen and that I could study when I wished at the pace I wished.
- References on course pages to external subject matter.
- Use of search engines/tools, Listserv, News and learning how the Net is built up.
- References to other sources and guidance and support.
- Access to academic articles and extracurricular subject matter.

Specific viewpoints on the course content and course specific experiences:
- That I learned to make Home Pages.
- Making web pages.
- Understand the whole system from making a WWW page to the server.
- Specific subject matter.
- HTML Programming (2).

General viewpoints:
- The course was very instructive and informative; the responses from the tutor were very good; I have learned a lot and I am well prepared for my next course.
- The reason why I took the course on the web was that it clashed with other courses (at the Polytechnic College); I find this form of teaching very good and wish to take similar courses at a later stage.
- WWW is a very attractive form of study; to me the main difference from correspondence study is that a) I am more comfortable with keyboard than pen and paper, b) I as tutor or student can do without stamps and post offices, c) I as tutor/student can study/teach when I wish and where in the world I am (`Tutor in Distance Education').

Which elements did you miss or find that could have been better exploited in your WWW course?
Aspects concerned with group communication:
- The discussion list was used very little - both by myself and others.
- I have been little active myself on the net because of work pressure.

Dissatisfaction with tutors' activity, the quality and speed of tutor feedback:
- The tutors could have been more active, e. g. short «lectures» paced with normal study progression, I miss a class-room teacher's pace and introduction to theory, important points, explanations etc. I feel this would have helped me disciplining and structuring the studies.
- Tutor participation.
- The Unix course, no feedback or help from the tutor.
- Generally bad communication between tutor and student, it takes too long to get answers when having problems.
- Contact with the tutor on the course.
- More activity from the tutors; references to subject matter/software.
- The technical competence on the tutor side could have been better, e. g. how to send diagrams.
- More extensive comments on assignments submitted.
- Some tutors are slow with feedback, even when the students are late, it should not take a week to get feedback especially just before exams.
- Some of the tutors are all too slow with feedback and answers to questions; they could also comment more on what is good and what could be done better; or else textbooks in Norwegian would have been fine.
- The initiative and support could have been better from some of the tutors.
- I would have preferred a more active tutor on the course; the communication should go both ways.

Specific course content missing:
- I feel that the Unix part was too small.
- The Unix part was too difficult.
- Two of the courses lacked practical assignments.
- Differences between different HTML standards and differences between different browsers related to this.
- Some software was not available.

Technical difficulties:
- I missed ISDN; surfing on the net is slow with ordinary modem
- I missed a central competent person in computing at NKI who could help solve technical problems; I missed a person who had absolute power to instantly change course elements and solve problems in the course; I would have changed everything that forced collaboration with fellow students.
- The tutor should inform students which types of files he is able to receive, the standards must be open; one could have started a news group which could be used also from the University of Oslo Intranet.
- Some difficulties with e-mail format and file transfer.

Viewpoints on how the WWW was exploited:
- More systematic use of resources - too many links here and there
- The interactivity which the WWW opens up; the course was too `static' with little participation from the tutors; the discussion groups could have been used better; the tutors could have supplied lectures and `overhead transparencies'.
- There were few differences between ordinary courses and the WWW course; the main difference was the 4 assignments for submission; except this there were only text pages on the WWW which we also have in other subjects with lectures (student at the Polytechnic College taking Data Communication as a selected subject at a distance).
- Some `links' in the material were not available any more (taken away), which was a problem; I was not prepared to read English texts.
- Concerning the subject matter on home pages where you found material on the WWW, I feel that the course explanations could have been better, apart from this I find this a fine way of studying and hope that more subjects can be offered this way.
- Use tools like CHAT, NETMEETING, NETSHOW etc. Leave the `correspondence model' and create interaction between students and tutor throughout the course; pay tutors properly, tutor fees restrict their activity - good luck!

Viewpoints that traditional ways of learning are preferred:
- The personal contact and help you can have in a classroom (`Information Technology for Teachers').
- The dynamics which can take place in classrooms.
- I missed meeting others taking the same course, it would have been interesting to discuss pedagogy and the course design (The tutor in distance education).
- There should have been more support and help during the course. The examination was different from what we learned it should be like.
- I missed somebody pushing me to keep up pace of study; incentives to complete the course; unused to communicate with fellow students, even when stimulated to do so, I felt it easier to put questions to the tutor.
- Guidance in the form of lectures and examination advice, help to decide what parts are directly exam related, what is necessary and what is just interesting.
- Data communication is too technical to learn exclusively on the net for one with little background as myself, I need lectures as support.
- The promise of support during the course was not kept. Direct teaching would have been better in this course. The exam assignments should have been more similar to the assignments for submission (Student at the Polytechnic College).
- Necessary software should be more easy accessible; lecture based teaching gives better progression and makes it easier to discuss.

Some Comments
When asking specifically about positive and negative elements of courses and experiences during a course, the answers cannot be taken as representative for a general attitude of being satisfied or dissatisfied with the course or learning experiences. The answers may give, however, some indication of elements that should be scrutinised for possible changes, revisions and quality improvements.

As far as we can judge from the above viewpoints, quite a number of students express some dissatisfaction with the quality and speed of feedback from the tutor(s). It seems that these complaints relate to tutors in specific courses. The viewpoints confirm earlier findings (see e. g. Rekkedal & Paulsen 1997) that group discussion for different reasons is lacking. The reasons are many, e. g. students say that they do not take the time to write entries to the conferences, some would like the tutors to be more active in the discussion groups, while others specifically stress that they are not interested in group communication or collaboration between students.

Four of the respondents are actually full time students at the NKI Polytechnic College and take the course as a selective subject within their full time studies. These students seem not to be specifically satisfied, either with the tutor or with WWW based learning. One explanation is that they have not deliberately chosen WWW based distance learning themselves. All 4 of these students express directly that face-to-face study would have been a better solution for them.

Summary and Conclusions

This survey was carried out among 55 students who had completed one or more distance courses based on the WWW at NKI Distance Education during 1997. 34 students answered the questionnaire (65% response rate). The survey was carried out as one of the activities of the EU Leonardo project `Multi-Media WWW Kernel for Distance Education'. Similar surveys will carried out by the other partners in the project.

The focal themes of the evaluation were the following:

· the students attitudes towards WWW-based learning
· their situation concerning use of computers and computer technology
· characteristics of students - who are attracted to WWW courses (1997)
· specific viewpoints on learning on the WWW and the actual courses

According to the original project plan this first evaluation was to examine student experiences from the course 'The Tutor in Distance Education' (Norwegian version). However, as so few students had completed this course when the pilot evaluation was to take place, we decided to include all courses on the kernel in the survey.

The students in this survey seem to be experienced PC users. They spend many hours working at their PC every week, most of them use a Pentium computer, the majority know how to make pages in HTML (not surprisingly, as some of the courses are related to the field), and most of them had an e-mail address before they enrolled for the course.

Less than 40 percent have paid for the course totally by themselves. Around 30 percent have the course paid totally by their employer or by a grant, while another 30 percent are supported by their employer or other source besides paying by themselves.

The group consists of nearly half men (16) and half women (17) (1 unanswered). (Actually the response rate is higher among women, as the questionnaire was distributed to 29 men and 26 women.) The median age among the respondents was 34.3 years. Thus, there is no sign that the NKI WWW courses recruit younger persons than distance education courses in general, as could perhaps be expected. The students represent different kinds of occupations. Some are teachers (as one of the programmes included in the survey was specifically further education in information technology for teachers). The others represent both administrative and technical jobs. Their previous education is relatively high, more than 60 percent have completed 1 year or more of higher education.

Their reasons for enrolling in the course vary. The most frequently mentioned reason is that the competence acquired from the course is needed in their present job.

The general impression is that the students in this survey are very positive towards learning through distance courses taught on the WWW.

A large majority of the students agree that:
· it is possible to achieve excellence in courses that are taught on the WWW
· enrolment was facilitated because the course was on the WWW
· they would enrol again in a WWW based course
· they would encourage a colleague or friend to enrol in a WWW based course

The students generally disagree that:
· the learning from the course was damaged because it was on the WWW

Bibligraphy

Fritsch, H. (1997): Host Contacted, Waiting for Reply. Hagen: FernUniversität. http://www.fernuni-hagen.de/ZIFF/EVIRTXT.htm

Paulsen, M. F. (1992): From Bulletin Boards to Electronic Universities: Distance Education, Computer-mediated Communication, and Online Education. 1992. University Park, Pennsylvania: The American Centre for the Study of Distance Education.

Paulsen, M. F. & Rekkedal, T. (1996): Technology for Adult Learning in Norway Including a Case Study on the NKI Electronic College. In Adult Learning and Technology in OECD Countries.

Rekkedal, T. & Paulsen M. F. (1997): The Third Generation NKI Electronic College - A Survey on Student Experiences and Attitudes, Norway.

Rekkedal, T. (1997): `Fjernstudenter med suksess'. Evalueringsundersøkelse ved NKI Teknisk Fagskole våren 1997. Internal report.

Appendix 1

Bekkestua, januar 1998

Til NKI-studenter som har fullført kurs basert på WWW i annet halvår 1997

NKI Fjernundervisningen deltar i et EU Leonardo-prosjekt i samarbeid med Ericson Expert Systems, Irland og Universitet Roma 3 i Italia. Deltakerne samarbeider om utvikling av WWW-tjenester for fjernundervisning og utprøving av undervisning basert på WWW.

I den forbindelse gjennomfører vi også en spørreundersøkelse med felles spørsmål til studenter ved de tre samarbeidende institusjonene. Fordi disse institusjonene er ulike, er noen av spørsmålene kanskje mer relevante for studenter ved en institusjon enn for studenter ved de andre. Vi håper likevel at du vil ta deg tid til å besvare skjemaet så fullstendig som mulig.

For NKI Fjernundervisningen er det viktig å få tilbakemelding om studentenes erfaringer og synspunkter. Undervisning over Internett og WWW er fremdeles ganske nytt. Studentenes synspunkter er vårt beste grunnlag for kontinuerlige videreutvikling og forbedring av undervisningsopplegg, IT-systemer og teknisk støtte.

Vi ber deg returnere skjemaet så raskt som mulig i vedlagte svarkonvolutt. Hvis du skulle ha besvart spørreskjema om Internett-basert fjernundervisning tidligere, vil vi likevel være glade for å få svar fra deg.

Spørreskjemaet tar opp tre områder:
- Verdien av WWW-kurs som én type studietilbud
- Markedet for WWW-kurs
- Karakteristika ved studenter som velger denne studieformen

På forhånd takk.

Vennlig hilsen
NKI Fjernundervisningen

Torstein Rekkedal Morten Flate Paulsen
forskningsleder amanuensis Spørreundersøkelse om WWW-baert fjernundervisning

Del 1:

Svarene på dette spørreskjemaet gjelder kurset/kursene_________________________

1. På bakgrunn av min deltakelse i kurset/kursene mener jeg at det er mulig å få fullt læringsutbytte gjennom kurs som undervises på WWW.
_ Helt enig
_ Enig
_ Usikker
_ Uenig
_ Helt uenig

2. Det at kurset ble tilbudt over Internett/WWW gjorde det enklere for meg å begynne på kurset.
_ Helt enig
_ Enig
_ Usikker
_ Uenig
_ Helt uenig

3. Min læring i kurset ble dårligere fordi kurset ble undervist over Internet/WWW.
_ Helt enig
_ Enig
_ Usikker
_ Uenig
_ Helt uenig

4. Mine læringsresultater i kurset var like gode som i et ordinært kurs med lærerledet undervisning i klasse.
_ Helt enig
_ Enig
_ Usikker
_ Uenig
_ Helt uenig

5. Erfaringene mine tilsier at jeg gjerne kunne ta et WWW-kurs ved en senere anledning.
_ Helt enig
_ Enig
_ Usikker
_ Uenig
_ Helt uenig

6. Jeg anbefaler gjerne en kollega eller venn å ta kurs basert på undervisning over WWW.
_ Helt enig
_ Enig
_ Usikker
_ Uenig
_ Helt uenig

Del 2:
7. Hvor mye tid brukte du ved en dataskjerm siste uke (inkludert arbeid, studier, fritid)?
_ Ingen
_ Mindre enn 5 timer
_ 6-10 timer
_ 11-20 tider
_ Mer enn 20 timer

8. Kan du konstruere eller endre en HTML-side?
_ Nei
_ Ja

9. Hvem betaler for kurset (inkludert tidsbruken)?
_ Jeg selv
_ Arbeidsgiver
_ Stipend/utdanningsfond
_ Kombinasjon av `jeg selv' og øvrige alternativer

10. Hva slags datamaskin har du hovedsakelig benyttet i kurset?
_ Pentium
_ 486
_ Mindre enn 486
_ Unix
_ Macintosh

11. Hvor lang tid tok det i gjennomsnitt fra du startet datamaskinen til du kom til kurssidene?
_ Svært raskt
_ Raskt
_ Sent
_ Svært sent
_ For sent til å kunne fullføre kurset

Del 3:
12. Din alder
_ < 24 år
_ 25-29 år
_ 30-34 år
_ 35-39 år
_ > 40 år

13. Kjønn
_ Kvinne
_ Mann

14. Yrke
_ Undervisningsstilling
_ Administrativ stilling
_ Teknisk stilling
_ Student
_ Arbeidsløs
_ Annet

15. Tidligere utdanning
_ Lavere enn 3-årig videregående skole/studiekompetanse
_ 3-årig videregående skole/examen artium
_ Utdanning utover videregående skole (ikke universitets-/høgskolestudier)
_ 1-3 år høgere utdanning
_ 4 år eller mer i høgere utdanning

16. Grunn til å begynne på kurset (sett bare ett kryss for den viktigste grunnen).
_ Behov for kunnskapene i nåværende jobb
_ Øke formell kompetanse
_ Muligheter for skifte av jobb
_ Interesse for faget
_ Interesse for undervisningsformen

17. Hadde du e-postadresse før du begynte på kurset?
_ Ja
_ Nei

18. Ser du det som en fordel at kursmaterialet på WWW også blir distribuert i trykt form?
_ Ja
_ Nei

19. Hvilke av elementene i WWW-kurset har du satt mest pris på?

20. Hvilke elementer savnet du mest eller synes du var svakt utnyttet i ditt WWW-kurs?

Andre kommentarer

Leonardo/MMWWWK2/leoneva2.doc