ABSTRACT
GHVFULEH DFWLYLW\ SDUWLFXODUO\ LQ WKH LQLWLDO VWDJHV DVVRFLDWHG UXJ$GPLQLVWUDWLRQ)'$7KLVIRFXVLVDFRQVHTXHQFHRIWKH WVDFWLYLVWVKDYHJLYHQWRWKH'UDL]HH\HWHVWXVHGWRWHVWSHUVRQDO IDOOXQGHUWKHMXULVGLFWLRQRIWKH)'$7KLVDWWHQWLRQOHGWRWKH RIWKHQHHGIRULQWHUDJHQF\DSSURDFKHVWRGHYHORSPHQWVUHODWLYH WLFXODUDQGDOWHUQDWLYHVLQJHQHUDO RPH WR OHDUQ WKDW WKHÀUVW WHVWPHWKRG LQ WR[LFRORJ\ WRGUDZ DOULJKWVDFWLYLVWVZDVQRWWKH'UDL]HWHVW7KH/'EHFDPHWKH
V LQ ,Q WKDW \HDU DQ )'$ZLGH VWHHULQJ FRPPLWWHH RQ VZDVRUJDQL]HG(DFKEXUHDXIRRGVGUXJVPHGLFDOGHYLFHV G RQ WKLV FRPPLWWHH DQG ZDV UHTXHVWHG WR SURYLGH D OLVW RI DWUHTXLUHXVHRIDQLPDOVDQGWKDWZHUHUHTXHVWHGRIUHJXODWHG HO\RURFFDVLRQDOO\7KHQXPEHURI DQLPDOVXVHGSHUGRVDJH
RUSHUWHVWZDVDOVRU UHTXLUHPHQWIRU/' UHVHDUFK DFWLYLWLHV R UHTXLUHGE\WKH)'$ HWF VXJJHVWLQJD UH 7KLVDFWLYLW\FXOPLQ LQFODULI\LQJWK
3ULRUWRWKH 'UDL]HH\H WHVWRQ W VRQQHO,QGXVWU\KDG WLYHVWRWKHH\HWHVW WKHDJHQF\ZDVZLOO DQGWKDWWKHJRYHUQP GHYHORSPHQWV,QGXV RQH IDFHW RI HYHQWXD IHGHUDO UHTXLUHPHQWV HDUO\ LQ WKH SURFHVV )'$SHUVRQQHOWRG
6HYHUDOFRVPHWLF LQ DQ XQGHUVWDQGLQJ SURFHGXUHVXQGHUGH 'UDL]HWHVW,WZDVD QRWEHXVHIXOIRUFRV UHDOL]HG WKDW LWZRX DQGJRYHUQPHQWRUJ DQGSURYLGHIHHGEDF UHTXLUH UHJXODWRU\ D SHUKDSVIRUJHDFRP SURYLGH LQGXVWU\ D DJHQFLHVZLWKRXWKD
$IWHUPXFKGLVFX ZKLFKKDGUHVSRQVLE FOHDUDQFHDQGRSLQLR PLVVLRQHURI WKH)' ZHUH VXSSRUWLYHRI (YDOXDWLRQ DQG 5HV 1DWLRQDO&HQWHUIRU7 WKH&HQWHU IRU'HYL ZDVUHDFKHGWKH(QY 6DIHW\&RPPLVVLRQ VHQWDWLYHVLQLW WRU\$OWHUQDWLYHV*UR
HTXHVWHG5HVSRQGHUVZHUHDVNHG´ZKHWKHUWKHUHZDVDQLPSOLHG
GDWDµDQG´ZKHWKHUDOOLQGLYLGXDOVDVVRFLDWHGZLWKUHJXODWRU\ I WKHLU GLYLVLRQRIÀFH ZHUH DZDUH WKDW WKH /' WHVW ZDV QRW µ,IWKHUHZHUHDQ\GRFXPHQWVJXLGHOLQHVUHFRPPHQGDWLRQV TXLUHPHQW IRU WKHFODVVLFDO/' FRSLHVZHUH WREHSURYLGHG DWHGLQDSROLF\VWDWHPHQWEHLQJLVVXHGLQ WKH)HGHUDO5HJLVWHU H)'$·VSRVLWLRQUHODWLYHWRWKLVWHVW SROLF\VWDWHPHQWPRUHDQGPRUHDWWHQWLRQZDVGLUHFWHGWRWKH KHSDUWRIDQLPDO ULJKWVDFWLYLVWVDQGFRQVHTXHQWO\)'$SHU DOVREHJXQWRGHYRWHUHVRXUFHVWRWKHGHYHORSPHQWRIDOWHUQD )'$UHSUHVHQWDWLYHVLQGLFDWHGLQPHHWLQJVDQGV\PSRVLDWKDW LQJ WRKROGGLVFXVVLRQVZLWK WKRVHGHYHORSLQJDOWHUQDWLYH WHVWV HQWZDQWHGWREHFRQVXOWHGDQGEHFRPHDQLQWHJUDOSDUWRIWKH WU\UHFRJQL]HGWKDWWKHGHYHORSPHQWRIDQDOWHUQDWLYHZDVRQO\ O DFFHSWDQFH DQG WKDW VXFK DQ DOWHUQDWLYHKDG WRPHHW FHUWDLQ 7KXV LW ZRXOG EH EHQHÀFLDO WR OHDUQ RI WKRVH UHTXLUHPHQWV RI GHYHORSPHQW ,QYHVWLJDWRUVZHUH HQFRXUDJHG WRPHHWZLWK LVFXVVWKHLUSODQV FRPSDQLHVPHWZLWK)'$SHUVRQQHO7KHVHGLVFXVVLRQVUHVXOWHG EHWZHHQ )'$ SHUVRQQHO DQG LQGXVWU\ WKDW LQ JHQHUDO WKH YHORSPHQWFRXOGEHXVHGDVVFUHHQVEXWFRXOGQRW UHSODFH WKH OVRXQGHUVWRRGWKDWWKHPHWKRGVZHUHSURGXFWVSHFLÀFDQGPD\ PHWLFVLQJHQHUDO$IWHUVHYHUDOVXFKPHHWLQJV)'$SHUVRQQHO OGEHXVHIXO WR H[SDQGGLVFXVVLRQV WR LQFOXGHRWKHU UHJXODWRU\ DQL]DWLRQV7KLVZRXOGDOORZ WKHDJHQFLHV WR FRPPHQW MRLQWO\ NWRLQGXVWU\RQDQDSSURDFKRUDVSHFLÀFWHVW7KLVZRXOGDOVR JHQFLHV WR EURDGHQ WKHLU YLHZRI WKH DSSOLFDWLRQRI D WHVW DQG PRQDSSURDFKWRLWVIXUWKHUGHYHORSPHQW$GGLWLRQDOO\LWZRXOG EHWWHU XQGHUVWDQGLQJ RI WKH SKLORVRSKLHV DQG SRVLWLRQV RI WKH YLQJWRRUJDQL]HVHSDUDWHPHHWLQJVZLWKHDFK*UHHQ
ESTABLISHMENT OF IRAG
VVLRQZLWKLQWKH&HQWHUIRU)RRG6DIHW\DQG$SSOLHG1XWULWLRQ LOLW\IRUFRVPHWLFVDQGEHIRUHFRQWDFWLQJRWKHUIHGHUDOHQWLWLHV QV UHJDUGLQJVXFKDQDSSURDFKZHUHREWDLQHGIURPWKH&RP $DQGRWKHU FHQWHUV UHVSHFWLYHO\&RPPHQWV IURPDOO FHQWHUV WKH HIIRUW5HVSRQVHVZHUH UHFHLYHG IURP WKH&HQWHU IRU'UXJ HDUFK WKH &HQWHU IRU %LRORJLF (YDOXDWLRQ DQG 5HVHDUFK 7KH R[LFRORJLFDO5HVHDUFKWKH&HQWHUIRU9HWHULQDU\0HGLFLQHDQG
FHVDQG5DGLRORJLF+HDOWK$IWHUFRQFXUUHQFHZLWKLQ WKH)'$ LURQPHQWDO3URWHFWLRQ$JHQF\(3$DQGWKH&RQVXPHU3URGXFW &36&ZHUHFRQWDFWHG$W WKHÀUVWPHHWLQJRIDJHQF\UHSUH ZDVGHFLGHGWRQDPHWKHRUJDQL]DWLRQWKH,QWHUDJHQF\5HJXOD XS,5$*5LFKDUG+LOO0'RIWKH(QYLURQPHQWDO3URWHFWLRQ
$JHQF\DJUHHG WR VH *UHHQRIWKH)'$5 IHGHUDORUJDQL]DWLRQV 7UDQVSRUWDWLRQUHSUHV 6DOHP WKH 1DWLRQDO :LOOLDP6WRNHVDQG E\6XUHQGHU$KLU1 ZDVDOVRLQYLWHGWRE UHVSRQVLELOLW\IRUDOW
7KHSXUSRVHRIWK WRZKROHDQLPDOWR[L RIFRPPRQLQWHUHVWWR SDUWLFLSDWHLQWKHFRO IRU WKH HYDOXDWLRQ R DFFHSWDQFHDQDO\]H VFLHQFH DQG IXQFWLR DFDGHPLDDQGSXEOLF KDUPRQL]DWLRQ%UDG
7KHÀUVWZRUNVKR RQWKH'UDL]HWHVW7K IUDPH7KLVZ IRUDGGLWLRQDOSODQQLQ ZDVDGLVFXVVLRQRI´ *UHHQ
7KHVHFRQGZRU 0HWKRGV 3URSRVDOV ²LQ:DV LQYLYRWHVWPHWKRGV VWDJHRIGHYHORSPHQ SODQQHGIRUWKHLQYL GLIIHUHQW WHVWLQJ DQ PLQLPL]HGGLIIHUHQF VFLHQWLÀFFRPPXQLW\ PHQWSURFHGXUHV
$WWKLVZRUNVKRS IRUWKH(\H,UULWDWLRQ WLRQµE\3DPHOD+XUO ´7KH 8VH RI /RZ9 ´1XPEHURI$QLPDO ,UULWDWLRQ7HVWVµE\: (YDOXDWLRQDQG&ODVV WKDW´ZLWKUHVSHFWWR
UYH DV FKDLUPDQ&KDUWHUPHPEHUVRI WKHJURXSZHUH6LGQH\ LFKDUG+LOORIWKH(3$DQG.DLODVK*XSWDRIWKH&36&2WKHU ZHUHLQYLWHGWRMRLQ7KRVHDJHQFLHVZHUHWKH'HSDUWPHQWRI HQWHGE\*HRUJH&XVKPDFWKH86$UP\UHSUHVHQWHGE\+DUU\ ,QVWLWXWH RI (QYLURQPHQWDO +HDOWK 6FLHQFHV UHSUHVHQWHG E\ WKH2FFXSDWLRQDO6DIHW\DQG+HDOWK$GPLQLVWUDWLRQUHSUHVHQWHG HLO:LOFR[RI WKH&HQWHU IRU9HWHULQDU\0HGLFLQHRI WKH)'$ HFRPHDPHPEHUJLYHQWKHIDFWWKDWWKHFHQWHUKDGWKHPDMRU HUQDWLYHVZLWKLQWKH)'$ HJURXSZDVWRSURYLGHDIRUXPIRUWKHGLVFXVVLRQRIDOWHUQDWLYHV FRORJLFDOWHVWPHWKRGV7KHDGKRFJURXSZDVWRLGHQWLI\LVVXHV WKHVFLHQWLÀFFRPPXQLW\LQERWKWKHSULYDWHDQGSXEOLFVHFWRUV OHFWLRQDQGH[FKDQJHRILQIRUPDWLRQDVVLVWLQVHWWLQJSULRULWLHV I LQ YLYR DQG LQ YLWUR PHWKRGV GHYHORSPHQW YDOLGDWLRQ DQG LQIRUPDWLRQDQGPDNHUHFRPPHQGDWLRQVFRQVLVWHQWZLWKJRRG Q DV D OLDLVRQ ZLWK LQWHUQDWLRQDO UHJXODWRU\ ERGLHV LQGXVWU\ LQWHUHVWJURXSVWRIDFLOLWDWHLQLWLDWLYHVWKDWOHDGWRLQWHUQDWLRQDO ODZDQG:LOFR[
IRAG WORKSHOPS
SZDVKHOGLQ6HSWHPEHURILQ$UOLQJWRQ9$DQGIRFXVHG HVWDWHRIDOWHUQDWLYHVZDVGLVFXVVHGLQWKHFRQWH[WRIWKDWWLPH RUNVKRS UHSUHVHQWHGDQ LQLWLDO HIIRUW LQWHQGHG WR VHW WKH VWDJH JDQGIRFXV5HSUHVHQWDWLYHRIWKHSUHVHQWDWLRQVDWWKLVPHHWLQJ UHJXODWRU\FRQVLGHUDWLRQVIRUDOWHUQDWLYHVWRWKH'UDL]HH\HWHVWµ
NVKRSZDVWLWOHG´:RUNVKRSRQ8SGDWLQJ(\H,UULWDWLRQ7HVW IRU 5HJXODWRU\ &RQVHQVXVµ DQG ZDV KHOG RQ 6HSWHPEHU KLQJWRQ'&7KHRULJLQDOLQWHQWZDVWRGLVFXVVLQYLWURDQG ,WEHFDPHFOHDUWKDWVSHFLÀFLQYLWURPHWKRGVZHUHQRWDWD WWREHFRQVLGHUHGDWWKDWWLPH7KXVDVHFRQGZRUNVKRSZDV WURPHWKRGV7KHSXUSRVHRIWKHÀUVWZRUNVKRSZDVWRLGHQWLI\ G HYDOXDWLRQ SUDFWLFHV GHYHORS SURSRVDOV IRU FKDQJH WKDW HVLQWKRVHPHWKRGVDQGGHWHUPLQHDUHDVRIFRQVHQVXVLQWKH 7KXVWKHHPSKDVLVZDVE\GHIDXOWRQUHGXFWLRQDQGUHÀQH
VHYHQSDSHUVZHUHSUHVHQWHG´&ULWHULDIRU,Q9LWUR$OWHUQDWLYHV 7HVWµE\6LGQH\*UHHQ´6FUHHQLQJ3URFHGXUHVIRU(\H,UULWD H\´8VHRI2SKWKDOPLF7RSLFDO$QHVWKHWLFVµE\9DQ6HDEDXJK
ROXPH 'RVLQJ LQ WKH (\H ,UULWDWLRQ 7HVWµ E\ /DUN /DPEHUW V IRU6HTXHQWLDO7HVWLQJµE\ -DQHW6SULQJHU ´6FRULQJ IRU(\H LOH\&KDPEHUV DQG´$Q(\H ,UULWDWLRQ7HVW3URWRFRO DQGDQ LÀFDWLRQ6\VWHPµE\.DLODVK*XSWD7KHZRUNVKRSFRQFOXGHG UHGXFWLRQDQGUHÀQHPHQWWKHVWURQJHVWFRQVHQVXVZDVUHODWHG
WRXVHRIWKHWZRVWDJ DQGWKHQDWXUHDQGFR WHVWSURYRNHGWKHVK LQIRUPDWLRQLVORVWZ HOLPLQDWLRQRID¶FRU WLFHVDQGFRQYHQWLRQ WHVWHLWKHUDVQXPE WRWDODFLGLW\RUDONDO SURFHHGLQJVRIWKHZ 7R[LFRORJ\9ROXPH GLVFXVVLRQRIWKHZR
7KHWKLUGZRUNVK WLWOHG ´:RUNVKRS RQ $QLPDO$OWHUQDWLYHV DSSURYDODQGDFFHSWD UHWURVSHFWLYH UHYLHZ JURXSVZDVFRQGXFWH RILQYLWURDOWHUQDWLY ZDVFRQFOXGHGWKDW´ RFXODU LUULWDQF\ WHVW 'UDL]HZHUHEHLQJX SUDFWLFH VRPHPRGH WHVWLQJSURYLGHG WKH ,W ZDV DOVR FRQFOXG PHFKDQLVWLFHQGSRLQ VXEVWDQFHVDWKLUGS YDOLGDWLRQDQGLQWHUQ DFFHSWDQFH EH JLYHQ SXEOLVKHGLQRQHYR WKHUHDGHULVUHIHUUHG
7KHIHGHUDOHPSO RUJDQL]LQJV\PSRVLD UHVSRQVLELOLWLHV6XS 6LQFHLWVLQFHSWLRQWK $VFRPSHWLWLRQIRUV ÀFXOW IRU WKLVFRKRUW WKHJURXSDQGWRVHH
7KHHVWDEOLVKPHQ RI$OWHUQDWLYH0HWKR RI WKH ,5$* ,WV P HVWDEOLVK FULWHULD DQ WR[LFRORJLFDOWHVWLQJ
HWHVWLQJLQZKLFKRQO\RQHRUWZRDQLPDOVZHUHVWXGLHGLQLWLDOO\ QVLVWHQF\RIWKHÀQGLQJVGLFWDWHGIXUWKHUWHVWLQJWKHORZYROXPH DUSHVWGHEDWHDQGSDUWLFLSDQWFRQVHQVXVZDVQRWHYLGHQWPXFK KHQWKHQXPEHURIJUDGHVRUFODVVHVRILUULWDQWVLVUHGXFHGWKXV URVLYHFODVV·ZRXOGUHTXLUHPRUHGLVFXVVLRQLQWHUQDWLRQDOSUDF VSUHYHQWHGDFRQVHQVXV UHJDUGLQJ WKHGHÀQLWLRQRIDSRVLWLYH HURIDQLPDOVUHVSRQGLQJRUDV WKHPHDQVFRUHDQGWKHXVHRI LQLW\DVDVXEVWLWXWHIRUS+UHTXLUHGIXUWKHUVWXG\µ7KHHQWLUH RUNVKRSZHUHSXEOLVKHGLQRQHYROXPHRI)RRGDQG&KHPLFDO DQGWKHUHDGHULVUHIHUUHGWRWKDWSXEOLFDWLRQIRUDIXOO UNVKRS RSZDVKHOGLQ1RYHPEHULQ:DVKLQJWRQ'&DQGZDV (\H ,UULWDWLRQ7HVWLQJ 3UDFWLFDO$SSOLFDWLRQV RI1RQ:KROH µ7KHJRDORIWKHZRUNVKRSZDVWRVHWDFRXUVHIRUWKHVFLHQWLÀF QFHRIQRQ²ZKROHDQLPDODOWHUQDWLYHVWRWKH'UDL]HH\HWHVW$ RI WKHQ H[LVWLQJ LQ YLWUR DQG LQ YLYR GDWD E\ H[SHUWZRUNLQJ GZLWKWKHLQWHQWWRH[DPLQHWKHVWDWXVRIDSUDFWLFDODSSOLFDWLRQ HVXVHGWRSUHGLFWH\HLUULWDWLRQ%UDGODZDQG:LOFR[,W GDWDZDVLQVXIÀFLHQWWRVXSSRUWWKHWRWDOUHSODFHPHQWRILQYLYR LQJ ZLWK LQ YLWURPHWKRGV DOWHUQDWLYH PHWKRGV WR WKH LQ YLYR VHGH[WHQVLYHO\E\ LQGXVWU\DV VFUHHQVEDVHGRQ WKHQFXUUHQW OVPD\ KDYH WKH SRWHQWLDO WR UHGXFH WKH QHHG IRU QHZ DQLPDO \DUHYDOLGDWHGDQGFRQGXFWHGXQGHUZHOOGHÀQHGFRQGLWLRQVµ HG WKDW ´EDVLF UHVHDUFK QHHGHG WR EH HQFRXUDJHG WR LGHQWLI\ WVIRUKXPDQRFXODULQMXU\WHVWEDWWHULHVLGHQWLÀHGWRWHVWQRYHO DUW\WDNHWKHOHDGWRLGHQWLI\SURPLVLQJPHWKRGVDQGWRIDFLOLWDWH DWLRQDOKDUPRQL]DWLRQRIPHWKRGGHYHORSPHQWYDOLGDWLRQDQG KLJK SULRULW\µ7KH HQWLUH SURFHHGLQJV RI WKHZRUNVKRSZHUH OXPHRI)RRGDQG&KHPLFDO7R[LFRORJ\9ROXPHDQG WRWKLVSXEOLFDWLRQIRUDIXOOGLVFXVVLRQRIWKHZRUNVKRS
EVOLUTION OF ICCVAM
R\HHVSDUWLFLSDWLQJLQYDULRXV,5$*DFWLYLWLHVVXFKDVPHHWLQJV DQGSDUWLFLSDWLQJLQH[SHUWZRUNJURXSVDOOKDGRWKHUSULPDU\ SRUWRIWKHVHLQGLYLGXDOVZDVFUXFLDOWRWKHVXFFHVVRIWKHJURXS HUHKDGEHHQQRIRUPDOEXGJHWWRVXSSRUWWKHJURXS·VDFWLYLWLHV FDUFHUHVRXUFHVPDWHULDOL]HGLWEHFDPHLQFUHDVLQJO\PRUHGLI WRGHYRWH WKH WLPHDQGHIIRUWQHHGHGWRFRQWLQXH WKHZRUNRI NDJHQF\DQGRWKHUIXQGLQJVRXUFHV
WRIWKH,QWHUDJHQF\&RRUGLQDWLQJ&RPPLWWHHIRUWKH9DOLGDWLRQ GV,&&9$0LQZDVYLHZHGDVDFRPSOHPHQWWRWKHZRUN LVVLRQ ZDV WR GHYHORS DQG YDOLGDWH QHZ WHVW PHWKRGV DQG WR G SURFHVVHV IRU WKH YDOLGDWLRQ DQG UHJXODWRU\ DFFHSWDQFH RI PHWKRGV$OWKRXJKQHYHUIRUPDOO\VWDWHG ,EHOLHYHWKH,5$*
YLHZHGLWVHOIDVFRQF LVVXHVVXFKDVH\HDQ WR[LFRORJLFDOPHWKRG DVWKHQFRPSRVHGR FLHV WKDWJHQHUDWHX DVVHVVPHQWSXUSRVHV WKH,&&9$0$VWKH RI(QYLURQPHQWDO+H HQWWKDWPXFKRIWKH WKH,&&9$07KHUH ZLWKLQWKHIHGHUDOJRY RI WKH ,5$*ZDV J DFFHSWHGDVWKHJURX GLQDWLQJDFWLYLWLHVUH
, WKLQN LW LV IDLU HVWDEOLVKPHQWRIWKH LQLWLDOO\ EHQHÀWHG WU H[HFXWHGWKHSURJUDP
HQWUDWLQJRQZKDWZHUHFRQVLGHUHGPDMRUUHJXODWRU\DOWHUQDWLYH GVNLQWHVWLQJ7KHUHKDGQRWEHHQPXFKWKRXJKWJLYHQWRRWKHU V7KH,&&9$0·VPDQGDWHZDVEURDGHU7KH,&&9$0LVQRZ IUHSUHVHQWDWLYHVIURPIHGHUDOUHJXODWRU\DQGUHVHDUFKDJHQ VHRUSURYLGH LQIRUPDWLRQ IURP WR[LFLW\ WHVWPHWKRGV IRU ULVN 1,(+6$OPRVWDOOPHPEHUVRI WKH ,5$*VHUYHGRQ ,&&9$0VXSSRUWHGE\UHVRXUFHVIURPWKH1DWLRQDO,QVWLWXWH DOWK6FLHQFHVEHJDQWRDVVHPEOHLWVSURJUDPLWEHFDPHDSSDU ZRUNFRQWHPSODWHGE\WKH,5$*FRXOGEHDFFRPSOLVKHGXQGHU FHUWDLQO\VHHPHGWREHQRUHDVRQIRUWKHH[LVWHQFHRIWZRJURXSV HUQPHQWKDYLQJVXFKFORVHO\UHODWHGDFWLYLWLHV7KXVWKHDFWLYLW\ UDGXDOO\ GLPLQLVKHG DQG DIWHU VRPH WLPH WKH ,&&9$0ZDV SZLWKLQWKHIHGHUDOJRYHUQPHQWKDYLQJUHVSRQVLELOLW\IRUFRRU ODWHGWRDQLPDODOWHUQDWLYHV WR VWDWH WKDW WKHZRUN RI WKH ,5$*ZDV WKH VWLPXOXV IRU WKH ,&&9$0DQGWKDWPXFKRIWKHZRUNRIWKH,&&9$0DWOHDVW HPHQGRXVO\ IURP WKH H[SHULHQFHRI WKRVHZKRRUJDQL]HGDQG VRI,5$*
REFERENCES
