Hanyoyin Ƙarfafa Rubutu Gabaɗaya don Rarraba Rubutun Gajere: Nazarin Kwatance

2.1 Hanyoyin Ƙarfafa Gabaɗaya

Takardar ta kimanta manyan dabarun ƙarfafa guda huɗu:

• Ƙarfafa na Tushen WordNet: Yana maye gurbin kalmomi da ma'anoni daga ma'ajin kalmomin WordNet. Yana buƙatar albarkatun harshe.
• Ƙarfafa na Tushen Word2Vec: Yana maye gurbin kalmomi da maƙwabtansu mafi kusa a cikin sararin haɗa kalmomin Word2Vec da aka riga aka horar. Ya fi dogaro da bayanai kuma ba shi da alaƙa da harshe.
• Fassarar Zagaye: Yana fassara rubutu zuwa wani harshe sannan ya dawo da shi zuwa asali, yana sake bayyana abun ciki. Yana amfani da APIs na fassara na waje (misali, Google Translate).
• Mixup: Wata fasaha ta tsari da ke ƙirƙirar samfuran horo na zahiri ta hanyar haɗa siffofi da alamun shigarwa: $\tilde{x} = \lambda x_i + (1-\lambda)x_j$ da $\tilde{y} = \lambda y_i + (1-\lambda)y_j$, inda $\lambda \sim Beta(\alpha, \alpha)$. Wannan yana ƙarfafa iyakokin yanke shawara masu santsi.

2.2 Tsarin Gwaji

An gudanar da gwaje-gwaje ta amfani da samfurin koyon zurfi don rarraba rubutu. An kwatanta tushe (babu ƙarfafawa) da kowace hanyar ƙarfafa daban-daban da kuma haɗe da mixup. An auna aiki ta amfani da daidaitaccen daidaiton rarrabuwa da maki F1, tare da mai da hankali kan rage yawan horo.

3.1 Kwatancen Aiki

Sakamakon, wanda za a iya gani a cikin ginshiƙi mai kwatanta maki F1 a cikin hanyoyi, ya nuna cewa:

• Ƙarfafa Word2Vec ya yi aiki kusan kamar Ƙarfafa WordNet, yana mai da shi madadi mai ƙarfi lokacin da albarkatun harshe ba su samuwa.
• Fassarar Zagaye, ko da yake yana da tasiri, an tauye shi da matsalolin kudi da samun dama, musamman ga yanayin ƙarancin albarkatu. Wannan ya yi daidai da damuwar da aka taso a cikin al'ummar NLP game da dogaro ga APIs na mallaka.
• Duk hanyoyin ƙarfafa rubutu sun inganta aiki fiye da tushe, suna tabbatar da ƙimar samar da bayanan roba don NLP.

3.2 Tasirin Tsarin Mixup

Wani muhimmin binciken shine tasirin haɗin gwiwa na mixup. Lokacin da aka yi amfani da shi a saman kowace hanyar ƙarfafa rubutu, mixup ya ci gaba da ba da ƙarin haɓaka aiki kuma ya rage yawan horo sosai, kamar yadda aka tabbatar da ƙaramin tazara tsakanin horo da asarar tabbaci. Wannan yana nuna mixup yana aiki azaman mai tsari mai ƙarfi wanda ke tilasta halayen layi tsakanin azuzuwan, yana inganta ƙaddarawa.

4.1 Tsarin Lissafi

Jigon ƙarfafa Word2Vec ya ƙunshi nemo manyan kalmomi k masu kama da kalmar da ake nufi $w$ a cikin sararin haɗawa, yawanci ana amfani da kamancen cosine:

$\text{sim}(w, w') = \frac{\mathbf{v}_w \cdot \mathbf{v}_{w'}}{\|\mathbf{v}_w\| \|\mathbf{v}_{w'}\|}$

inda $\mathbf{v}_w$ shine wakilcin vector na kalmar $w$. Ana maye gurbin kalma da ɗaya da aka zana bazuwar daga cikin maƙwabtanta k mafi kusa tare da ƙayyadadden yuwuwar $p_{replace}$.

Tsarin mixup don tarin samfura shine:

$\tilde{X} = \lambda X_i + (1-\lambda) X_j$

$\tilde{Y} = \lambda Y_i + (1-\lambda) Y_j$

inda $\lambda \sim \text{Beta}(\alpha, \alpha)$, $\alpha \in (0, \infty)$. Ƙaramin $\alpha$ (misali, 0.2) yana ƙarfafa ƙarin haɗawa.

4.2 Misalin Tsarin Nazari

Yanayi: Wani kamfani mai farawa yana son gina mai rarraba ra'ayi don tweets na abokin ciniki amma yana da misalai 1,000 kawai da aka yiwa lakabi.

Aikace-aikacen Tsarin:

1. Kimanta Albarkatu: Kamfanin ba shi da kasafin kuɗi don APIs masu tsada kuma ba shi da ma'ajin ma'anoni da aka tsara. ⇒ An kawar da WordNet da fassarar zagaye.
2. Samfuri & Tushe: Horar da samfurin LSTM ko Transformer mai sauƙi (misali, ƙaramin BERT) akan samfuran 1,000 a matsayin tushe.
3. Bututun Ƙarfafa:
   • Yi horo ko saukar da samfurin Word2Vec akan babban tarin rubutu na gabaɗaya (misali, bayanan Twitter).
   • Aiwatar da ƙarfafa na tushen Word2Vec akan saitin horo, yana samar da sigogin ƙarfafa 2-3 a kowane samfurin asali.
   • Aiwatar da mixup yayin horo akan tarin asali + ƙarfafa.
4. Kimantawa: Kwatanta daidaiton tabbacin tushe da samfurin ƙarfafa+ mixup. Lura da tazarar asarar horo/tabbaci don duba rage yawan horo.

Wannan tsarin yana ba da fifikon inganci da kudi da aiki, yana bin fahimtar takardar kai tsaye.

5.1 Nazari na Asali: Jagorar Mai Aiki na Ƙarfafa NLP

Babban Fahimta: Aikin Marivate da Sefara ba game da sabuwar hanyar ƙirƙira ba ne, amma game da binciken gaskiya da ake buƙata sosai ga masana'antar NLP. A cikin fagen da ke damuwa da samfuran da suka fi girma, sun sake mai da hankali kan tattaunawa akan wani mahimmin matsalar toshewa: ƙarancin bayanai. Babban ra'ayinsu—cewa sauƙi, ƙarfafa duniya (Word2Vec) da aka haɗa da mixup shine madadin aiki mai tasiri kuma mai sauƙin isa—magani ne mai ƙarfi ga zaton cewa kawai samfuran masu rikitarwa, masu fahimtar mahallin kamar BERT ne za su iya magance yunwar bayanai. Wannan yana maimaita falsafar da ke bayan takardun ƙarfafa hangen nesa na kwamfuta na asali, inda sauƙaƙan sauye-sauyen geometric suka tabbatar da ƙima sosai.

Kwararar Hankali & Ƙarfafawa: Hankalin takardar yana da aiki sosai. Ya fara ne daga ƙayyadaddun duniya (ƙayyadaddun lakabi), yana kimanta mafita akan bakan inganci da kudi, kuma ya isa ga shawara mai aiki bayyananne. Ƙarfafawa yana cikin ƙarfin kwatance. Ta hanyar sanya hanyoyin harshe (WordNet), dogaro da bayanai (Word2Vec), da hanyoyin sabis na waje (fassara) a kan juna, sun ba da shaida na zahiri ga abin da yawancin masu aiki suka yi zargin: ba koyaushe kuke buƙatar albarkatu masu ƙwarewa ba. Haɗin mixup yana da wayo musamman. Asalinsa daga hangen nesa (Zhang et al., 2018), nasararsa a nan yana jaddada ƙa'idar da za a iya canzawa: ƙarfafa haɗin layi tsakanin azuzuwan yana gina iyakokin yanke shawara masu ƙarfi, wani fahimta da ka'idar kwanan nan kan tasirin santsi na mixup (Carratino et al., 2020) ke goyon bayan.

Kurakurai & Damuwar da aka rasa: Nazarin, duk da haka, ya tsaya gaban iyakar hankali ta gaba. Yayin da suka yi wayo suna alamar farashin APIs na fassara, ba su cika fuskantar iyakokin haɗa kalmomin duniya ba. Kamancen "duniya" na Word2Vec na iya komawa baya—maye gurbin "banki" (kogi) da "cibiyar kuɗi" yana lalata ma'ana. Fagen ya taɓa zuwa ga haɗa kalmomin mahallin (ELMo, BERT). Wani babban rashi shine gwada ƙarfafa ta amfani da maye gurbin kalma na mahallin daga samfuran harshe da aka rufe, fasaha da yanzu ta zama gama gari (misali, Kobayashi, 2018). Bugu da ƙari, kimantawa ya taƙaita ga daidaiton rarrabuwa; sun rasa damar yin nazarin tasirin ƙarfafa akan daidaita samfur da kimanta rashin tabbas—mai mahimmanci don turawa.

Fahimta Mai Aiki: Ga masu aiki, abin da za a ɗauka yana bayyananne: Kafin kai ga samfurin da aka riga aka horar da shi mai girma ko API mai tsada, ƙarfafa sauƙi + mixup. Wannan haɗin gwiwa shine layin tsaron ku na farko a kan yawan horo a cikin ƙarancin bayanai. Ga masu bincike, takardar ta nuna gibin gaggawa: 1) Tsarin gwajin hanyoyin ƙarfafa mahallin da waɗannan na duniya, da 2) Haɓaka ma'auni na kimanta samfurin "mai fahimtar ƙarfafa" waɗanda suka wuce daidaito don auna ƙarfi ga maye gurbin ma'ana da sake bayyana. Gaba ba kawai a cikin ƙirƙirar ƙarin bayanai ba ne, amma a cikin ƙirƙirar bayani masu wayo, na dabarun waɗanda ke magance raunin takamaiman samfur.

5.2 Ayyuka na Gaba & Jagorori

Ka'idodin da aka zayyana suna da fa'ida mai faɗi kuma suna nuna zuwa ga jagorori da yawa na gaba:

• NLP na Harshe mai Ƙarancin Albarkatu: Ana iya horar da samfuran Word2Vec akan ƙananan tarin rubutu guda ɗaya, yana mai da wannan bututun ya dace da harsunan da ba su da kayan aikin harshe ko tallafin fassara.
• Daidaituwar Yanki: Ƙarfafa na iya taimakawa daidaita mai rarraba na gabaɗaya (misali, ra'ayi) zuwa takamaiman yanki (misali, dandalin likita) ta hanyar samar da rubutu na roba a cikin yanki, yana rage canjin rarraba.
• Haɗawa tare da Koyo Mai Ƙarancin Kulawa: Aikin gaba zai iya haɗa ƙarfafa sosai tare da tsarin horon kai ko daidaitaccen horo. Samfurin da aka horar da shi akan bayanan da aka ƙarfafa zai iya samar da alamun ƙarya don bayanan da ba a yiwa lakabi ba, waɗanda aka ƙara ƙarfafa su kuma a yi amfani da su don horo.
• Bambance-bambancen Mixup na Ci Gaba: Bincika manifold mixup ko mixup na matakin jumla a cikin sararin haɗawa na masu canzawa na zamani (kamar Sentence-BERT) zai iya haifar da ƙarin riba.
• Koyon Manufar Ƙarfafa ta atomatik: An yi wahayi ta AutoAugment a cikin hangen nesa, bincike zai iya mai da hankali kan koyon dabarun ƙarfafa mafi kyau (waɗanne kalmomi za a maye gurbin, da wace yuwuwar) kai tsaye daga bayanai.