A Study of IRMA Admissions

M.J. Arul
Institute of Rural Management, Anand

Introduction

Institute of Rural Management, Anand (IRMA) is an autonomous institution for promotion of professional management in rural development organisations. In pursuance of this mission, the Institute selects, prepares and places people who are to effectively manage the businesses of rural producers' co-operatives throughout India.

The mission began in 1979. It's 1983 now and over eighty graduates from the first two batches of the Institute have already been placed in various jobs. The third batch of 55 is nearing its graduation and placement. The fourth one of 63 is at the moment doing its first term of studies in the two-year postgraduate Programme in Rural Management (PRM). The admissions office is getting ready for the next haul. The cycle continues, year after year.

In the context of IRMA's admissions, it is certainly fitting and proper to ask as to what makes an effective rural manager, what makes a good candidate, and what makes a suitable PRM. Much of this information could be obtained from a comprehensive study which would enquire into the profiles and job contents of rural managers at large and relate the findings to the design and contents of the PRM as well as to the potential profiles of candidates for the Programme. Such a study should also undertake to examine the critical attitudes and skills required in the rural managers' domain of operation and determine the extent to which IRMA's Programme cultivates these attributes.

But that was not to be! A much less ambitious study was undertaken in August 1983, with the belief that even a limited analysis of the data available within the Institute would provide us with some useful feedback and also throw up pointers for future studies on the topic. The study sought answers to questions such as how do the demographic variables (on which information was available from the application blanks) relate to performances in the written test, the interview and the PRM? What is the predictive validity of the various measures used in the selection process as well as in the Programme vis-a-vis performance on the job after placement?

The study has, in several respects, revealed the obvious--a good enough finding, perhaps, to urge us to examine some of the assumptions we have gone by. There have also emerged a few indicators of what is often not so obvious.

Methodology

The first three batches of students constituted the sample and seventeen variables were identified for the study. The variables were: gender, caste, place of schooling, place of graduation, subject at graduation, parent's occupation, family income, school-final marks, college-final marks, written-test score, verbal score, math score, interview score, soft-course average, quanty-course average, over-all grade and product rating. Data on all these variables, save for the last one, were collected from the appropriate records available with the Institute's Admissions and Programme offices. Data for the last variable (product rating, i.e., how well the person has shaped up as a student at the Institute) were obtained from a sample of the faculty.

As a proxy for the on-the-job success of our alumni (an additional variable), the faculty (those in touch with the employers) were asked to identify names of alumni who were successful on the job and who were unsuccessful. Only those alumni were included in this analysis who were classified in either category by at least four of the five judges.

Data analysis was carried out by the statistical tools of correlation, analysis of variance, t-test and multiple regression.

Findings

The results of the study were summarised in eighteen Tables. A few selected findings are taken up here for discussion:

The rough-and-ready classification of successful versus unsuccessful alumni (Table 15) shows that those from richer families do better on the job. Does this mean the rich are more suitable for IRMA? After all, one should not have to be poor in order to be of help to them!

Before adopting such a point of view, however, let's take a look at a few more of the findings to see whether there are any intervening variables, possibly influencing performance.

Girls in our Programme, for example, come from significantly richer families than boys, but perform poorer at the Institute (p<0.10 on OGPA and p<0.01 on Faculty Rating: Table 1). An examination of performances of the students, classified on the basis of their parental income (Table 7), showed no differences (except in one case, discussed below), strongly indicating that income as such is not a determinant of performance.

The one case of difference is that students from richer families do poorly on the quantitative part of the written test. This finding is corroborated by a negative correlation between income figures and quantitative test score (Table C-1). This "anomoly" finds an explanation in Table 5, which shows that Engineering students come from families of much lower income than Arts students. And the mathematical abilities of engineers are well known to be superior to those of Arts graduates.

On classifying the candidates by their parents' occupation (Table 6), we find that wards of professionals and government officers, both of whom show a relatively high income, do better in our interview (p<0.01) than those whose parents are officers in the private sector with even a higher family income. Further, students with agriculture and veterinary science backgrounds come from relatively low-income families (Table 5), but obtain a faculty rating as high as that of others. Parental income or economic status is, thus, not the determinant of on-the-job performance.

Scores on the verbal portion of our Admissions test were found (Table C1) to correlate positively with the interview ratings (r = 0.44; p<0.01) as well as with the grades secured in the soft courses in the Programme (r = 0.29; p<0.05). Further analysis (Table C3) shows that scores on the verbal portion of the test correlate very highly with interview ratings, specially for candidates from metro cities (r = 0.72; p<0.01) and to a much less extent for the small towners (r = 0.33; p<0.05). The interview ratings, in turn, correlate positively with the overall performance (OGPA) in the Programme (r = 0.32; p<0.05). The higher the verbal skill, it appears, the better the chances of entering, as well as faring well at, IRMA. Is it a halo-pygmalion effect?

One is driven at this point to search the interview. Over and above the relationships discussed in the preceding paragraph, the interview results also show a positive correlation with the school marks and written-test scores. Degree marks, while they do show a correlation with school marks, do not correlate either with the interview ratings or with the written-test scores. This suggests that the Admissions test and the interview engage the general faculties of candidates rather than proficiencies in any specific discipline. (Subjects taught at the school level are more general than those at the college; students stream off into different disciplines at the college level).

Be that as it may, does the interview assess anything more than what has been done through other mechanisms prior to the interview? Subjecting the interview results to a multiple regression analysis with income, school marks, degree marks, verbal score and quantitative score, we get an R2 of 0.562. The remaining variance is sizeable and can be attributed to the interview. The return on investment in the interview would be appreciable, therefore, if it actually helped improve the chances of identifying the right candidates for the PRM. Four more regressions were run to test this.

Faculty rating revealed an R2 of 0.453 when it was regressed on income, school marks, degree marks, written test, interview and OGPA; the coefficient turned out to be 0.451 when interview was factored out. Similar analysis was done for OGPA as the dependent variate and the R2 encompassing interview was 0.399 and without interview 0.355. All this indicates that whatever was assessed in the interview turns out to be irrelevant and its contribution to the PRM is virtually nil. That the interview does not bring out anything new about the candidate is also corroborated by Tables 8 and 10. It may also be noted that the successful and the unsuccessful alumni do not differ on their interview scores (Table 15).

Why not drop the interview?

Should the interview be dispensed with, then? The results render it plausible to argue that doing away with the interview would not have affected the quality of students who were admitted. While considering this argument, it may also be borne in mind that all our assessments taken together explain only about 40 per cent of the performance in the PRM courses and 45 per cent of the faculty rating. Before deciding, therefore, on the continuance or cessation of the interview, it is imperative that we identify the factors which could account for the remaining variance in the faculty rating, which seems to be highly predictive of success on the job.

A qualitative analysis of the discussions with some of the employers, alumni and faculty members, pointed to the following critical attributes for success: One's general orientation to life, attitude towards the career available through IRMA's placement service and interpersonal relations with colleagues and superiors. Honesty and a sense of fairness may well be added to these, if our alumni are to be successful--honourably. The exact nature of these and other necessary characteristics may be clarified by further discussions with (or a study of) practising managers in the sector.

Then what?

Once the critical qualities are identified and agreed upon, how could IRMA ensure their presence in its graduates? One way would be to select candidates who already, at the time of their admissions, possess these attributes and maintain, during the more mundane training in the PRM, an atmosphere which will foster rather than choke off the precious personality traits. Alternatively people could be taken in on the basis of some objective criterion which--irrespective of its relationship to the ultimate goal--would represent a fair play to the competing aspirants and then offer the participants a programme so designed and administered as to actively inculcate in them the necessary personal qualities alongside of proficiency in the management science.

Thinking through either of the above alternatives stirs up, amidst the easily recognised problems of methodology in selection and training, the immense role required of the faculty. In the context of students' uncanny ways of sensing discrepancies between the public and private selves of teachers and given the psychological fact that lessons and messages that are picked up from the teachers are unfortunately not confined to what is said in the classroom, the priority seems to need to shift to the recruitment, selection and induction of the faculty. Howsoever inescapable and important this priority may be, it is beyond the scope of the present study. Returning, therefore, to the concerns of selecting candidates for the PRM, the reader may at this point like to take a look at a note, which was written over two and a half years ago (in 1980) and is, perhaps, relevant here.

Group discussion: An alternative

In spite of the biases that can and often do operate in personal assessments of another person, the interview in one form or another has been long recognised as a useful tool the world over. My 1980 note, as you may have seen, advocates certain sensitivity while conducting the interview. All the same, the note is not intended to undermine the difficulties involved.

Considering the kind of qualities to be assessed, it may be better for IRMA to adopt, in lieu of the current 10-20-25-minute sessions for individual candidates, a much longer (say, an hour or so long) group discussion which is to be keenly observed by a panel of faculty. Here a number of candidates are available for observation and assessment during the entire duration of the "interview"--even those who are silent verbally. The situation is also common for all the members of the group -- unlike in the individual interviews. The observors are better tuned in, too, freed from distractions which prevail in personal interviews: distractions such as having to think up, suitably phrase, rephrase and repeat questions, over and above the transactions, both intended and unintended, among the interviewers themselves. The optimum size of the group and the exact mechanics of this mechanism can be worked out, once the group discussion method is accepted in principle. (This method of selection was tried out in the next two years and the Admissions Office reported better results. The group discussion was subsequently discontinued for lack of people to conduct it.)

Entry level cut-off

As regards the initial criterion to quality for the Admissions test, it is difficult to determine on the basis of the present study as to what should be the optimal minimum, since school marks do not seem to influence performance at the Institute (Table C-1 and C-2), except perhaps in the case of candidates from small towns (Table C-3). In the light of these findings, the cut-off level which was recently raised should not really have any adverse effects--if the results in Table 15 is taken as not representing reality: unsuccessful alumni have higher marks at their school final than successful ones (p<.10). More reliable data will be available for this table after a good number of our alumni have had a reasonable length of time to prove their mettle on the job. At the moment, the only additional support Table 15 can have is from Table C-3: while students from metro cities and state capitals show hardly any correlation between their school marks and their performance at IRMA, those from minimetros register a negative correlation (-0.85) at p<.05.

The prescription of a qualifying minimum in the earlier academic records seems to be nothing more than a means of reducing numbers on an objective basis. If past academic records are accepted as having a bearing on performance at IRMA and later on the job, we may consider the advisability of using a double cut-off, with a lower limit and an upper limit. The validity or relevance of such a criterion is supported by (a) the client sector's inclination not to seek academically very bright people and (b) the indicative data on Table 15(ii): beyond a point, higher marks at school do not seem to contribute to better performance on the job.

The written test

No matter what the cut-off limits be, the criteria of school and college marks do not seem to serve any additional function in selection -- other than limiting the number of candidates allowed to sit for the test. Since IRMA cannot influence the contents of school/college tests and examinations, it cannot do anything to enhance the utility (predictive validity) of their marks. For purposes of improving the quality of selection at IRMA, the Admissions Test is more serviceable, depending on its contents, which should be subjected to an analysis of validity. The test items should also have appropriate levels of difficulty and discriminative ability, if the test is to be of consequential utility for IRMA.

Limitations

The study was conducted only on those who completed the PRM. Further, some of the categories were rather too small to be capable of rigorous statistical analysis. A comparative analysis of the different batches could have been useful for identifying any inter-batch variations in certain variables and their influences on performance, faculty rating, etc. But no batch-wise analysis was carried out. A future study of the topic will certainly be free of several constraints and inconveniences that were very much part of the present one and can get into analysis of selected variables straight away.
ANNEXURE: 1

A Note on the PRM Selection Interview

We are soon to interview candidates for potential rural managers. It may be useful to consider at this juncture that the selected candidates are not going to step into the job straight away; two full years of training, preparation or moulding is to precede their doing so. The job contents and contexts, therefore, of rural managers, while they must certainly inform our curriculum, must not become the direct criteria on which to select candidates for the PRM.

We need to select those who hold out a promise of learning most from our programme. Inferring from the fact that the candidate has been screened in for the interview, we may accept that s/he is good enough in the requisite scholastic abilities, checked out in the written test. In the interview, therefore, we may look for additional indicators of the candidate's suitability for th PRM. Aptitude for effective interpersonal relations, attitude towards the programme, honesty, clarity of thought, self-reliance and aptitude for living with uncertainty are some of the dimensions relevant in the context.

The interviewers must do their best to elicit from the candidate responses such as will reveal the latter's position with regard to the dimensions mentioned above. The key is to let the candidate do most of the talking at the interview. Following are questions that could serve as conducive stimuli to evoke a fair sample of responses:

It must be emphasised here that the verbal contents of answers obtained in response to the questions such as suggested above, are likely to be neither idiosyncratic nor characteristic of one or the other personality group; they are, therefore, incapable of any meaningful discrimination. And yet, attentive listening to the candidate's oral responses, coupled with attention to the metaverbal cues (such as facial expressions, eye-contact and gestures) that accompany the words, can help assess the candidate on dimensions of relevance to us.

The efficacy of the assessment may be enhanced if questions such as the following are run through the minds of the interviewers, during the course of the interview:

Back to where you left off.
APPENDIX: 1

Interview Rating Sheet

Date: ___________ Session number: ____

Please Note:

1. Rate the candidate on each of the dimensions out of 10 points.
2. Avoid the tendency to be consistent in your ratings across dimensions on the basis of the candidate's high or low score on a particular dimension of importance.


Name of
candidate
Clarity of
thought
EnthusiasmAuthenticity
and "SOB"*
Ability to cope
with uncertainty
Openness
and listening
Special comment
if any
1.
--
--
--
--
--
..........
2.
--
--
--
--
--
..........
3.
--
--
--
--
--
..........
4.
--
--
--
--
--
..........
5.
--
--
--
--
--
..........
* Sophisticated overt behaviour.

________________
Signature (Interviewer)

Back to where you left off | Go to other studies.