Sakamako Na Farko Akan Fassarar Injin Neural Na Larabci: Bincike Da Fahimta

Tsarin Abubuwan Ciki

1. Gabatarwa & Bayyani

Wannan takarda ta gabatar da aikace-aikacen farko na cikakke na Fassarar Injin Neural (NMT) akan Larabci, harshe mai wadataccen siffa da rikitaccen tsari. Yayin da NMT ta nuna babban nasara akan harsunan Turai, tasirinta akan Larabci bai kasance an bincika ba. Binciken ya gudanar da kwatancen kai-tsaye tsakanin daidaitaccen ƙirar NMT mai tushen kulawa (Bahdanau et al., 2015) da tsarin Fassarar Injin Ƙididdiga (SMT) mai tushen jumla (Moses). Binciken ya mai da hankali kan fassara ta kowane bangare (Larabci-zuwa-Turanci da Turanci-zuwa-Larabci), yana binciken tasirin muhimman matakan sarrafa kafin aiwatarwa na musamman ga Larabci kamar rarraba kalmomi da daidaita rubutu.

2. Tsarin Fassarar Injin Neural

Jigon tsarin NMT shine ƙirar mai rufe-mai buɗe mai tushen kulawa, wanda ya zama daidaitaccen tsarin ƙira na zahiri.

2.1 Tsarin Mai Rufe-Mai Buɗe

Mai rufewa, yawanci Cibiyar Sadarwar Neural Mai Maimaitawa (RNN) mai bangarori biyu, tana sarrafa jimlar tushe $X = (x_1, ..., x_{T_x})$ kuma tana samar da jerin ƙwayoyin mahallin $C = (h_1, ..., h_{T_x})$. Mai buɗewa shine ƙirar harshe ta RNN mai sharadi wacce ke samar da jerin manufa kalma ɗaya a lokaci guda, ta yin amfani da yanayinta na baya da kalmar da aka samar a baya.

2.2 Tsarin Kulawa

Tsarin kulawa yana ƙididdige jimillar nauyin ƙwayoyin mahallin mai rufewa a kowane mataki na buɗewa. Wannan yana ba ƙirar damar mai da hankali ga sassa daban-daban na jimlar tushe yayin da take samar da fassarar. Ƙwayar mahallin $c_{t'}$ a lokacin matakin mai buɗewa $t'$ ana ƙididdige ta kamar haka:

$c_{t'} = \sum_{t=1}^{T_x} \alpha_{t} h_{t}$

inda ake ƙididdige ma'aunin kulawa $\alpha_{t}$ ta hanyar cibiyar sadarwa mai ci gaba da gaba tare da ƙwaƙwalwar ajiya guda ɗaya ta tanh: $\alpha_{t} \propto \exp(f_{att}(z_{t'-1}, \tilde{y}_{t'-1}, h_t))$. A nan, $z_{t'-1}$ shine yanayin ɓoyayyen mai buɗewa na baya kuma $\tilde{y}_{t'-1}$ ita ce kalmar manufa da aka fassara a baya.

2.3 Tsarin Horarwa

Ana horar da dukan ƙirar ta kaiwa-zuwa-ƙarshe don haɓaka matsakaicin log-likelihood na sharadin fassarar manufa idan aka ba da jimlar tushe. Ana cimma wannan ta hanyar amfani da gangaren stochastic tare da baya-bayan baya ta lokaci (BPTT).

3. Tsarin Gwaji & Hanyoyin Bincike

3.1 Bayanai & Sarrafa Kafin Aiwatarwa

Binciken yana amfani da daidaitattun tarin bayanai na Larabci-Turanci. Wani muhimmin al'amari shine kimanta hanyoyin sarrafa kafin aiwatarwa daban-daban na rubutun Larabci, gami da rarraba kalmomi ta hanyar siffa (misali, raba ƙananan kalmomi da kari) da daidaita rubutu (misali, daidaita siffofin aleph da hamza), waɗanda aka sani suna da mahimmanci ga SMT na Larabci (Habash da Sadat, 2006).

3.2 Saitunan Tsarin

• Tsarin NMT: Ƙirar asali mai tushen kulawa (Bahdanau et al., 2015).
• Tushen SMT: Daidaitaccen tsarin tushen jumla da aka gina ta amfani da kayan aikin Moses.
• Masu Canji: Haɗuwa daban-daban na rarraba kalmomi da daidaitawa don Larabci.

3.3 Ma'aunin Kimantawa

Ana kimanta ingancin fassarar ta amfani da daidaitattun ma'auni na atomatik kamar BLEU, kwatanta aiki akan gwaje-gwajen cikin yanki da na waje da yanki don kimanta ƙarfin jurewa.

4. Sakamako & Bincike

4.1 Aikin Cikin Yanki

Tsarin NMT da SMT mai tushen jumla sun yi aiki daidai da juna akan gwaje-gwajen cikin yanki na kowane bangaren fassara. Wannan sakamako ne mai mahimmanci, yana nuna cewa ko da ƙirar NMT ta farko, "asali", za ta iya dacewa da aikin tsarin SMT da ya kafa a kan nau'in harshe mai kalubale.

4.2 Ƙarfin Jurewa Na Waje Da Yanki

Wani muhimmin binciken shine cewa tsarin NMT ya fi tsarin SMT girma sosai akan gwajin da ba na yanki ba don fassarar Turanci-zuwa-Larabci. Wannan yana nuna cewa ƙirar NMT suna koyon wakilci mafi gama-gari waɗanda ba su da rauni ga sauyin yanki, babbar fa'ida ce don aiwatarwa a duniyar zahiri inda bayanan gwaji sau da yawa suka bambanta da bayanan horo.

4.3 Tasirin Sarrafa Kafin Aiwatarwa

Gwaje-gwajen sun tabbatar da cewa daidaitaccen sarrafa kafin aiwatarwa na rubutun Larabci (rarraba kalmomi, daidaitawa) yana da tasiri mai kyau iri ɗaya akan tsarin NMT da SMT. Wannan yana nuna cewa waɗannan fasahohin suna magance ƙalubalen asali na harshen Larabci da kansa, maimakon kasancewa na musamman ga wani tsarin fassara na musamman.

5. Zurfin Binciken Fasaha & Ra'ayin Mai Bincike

Mahimman Fahimta: Wannan takarda ba kawai game da amfani da NMT akan Larabci ba ce; gwajin damuwa ne wanda ke bayyana fa'idar asali amma mahimmanci ta NMT: Mafi kyawun koyo da wakilci da gama-gari. Yayin da SMT ta dogara da daidaitaccen jeri da teburan jumla da aka ƙera da hannu, tsarin mai rufewa-kulawa-mai buɗewa na NMT yana koyon taswira mai ci gaba da sanin mahalli a ɓoye. Tazarar aikin da ba na yanki ba ita ce hujja. Tana gaya mana cewa wakilcin jijiyoyi na NMT yana ɗaukar ƙa'idodin harshe masu zurfi waɗanda ke wucewa ta yankuna, yayin da teburan ƙididdiga na SMT sun fi yawan haddacewa kuma suna da rauni.

Kwararar Ma'ana: Hanyar binciken marubutan tana da wayo. Ta hanyar riƙe sarrafa kafin aiwatarwa a tsaye da kuma tsayar da NMT "asali" da SMT "asali", sun ware gudummawar ƙirar asali. Gano cewa sarrafa kafin aiwatarwa yana taimaka wa duka biyu daidai fasaha ce mai ƙware—tana warware gardamar cewa duk wani nasarar NMT kawai saboda mafi kyawun daidaita rubutu ne. Sa'an nan hankali ya faɗi daidai kan iyawar ƙirar da ke ciki.

Ƙarfi & Kurakurai: Ƙarfinsa shine tsarin gwaji mai sarrafawa, bayyananne wanda ke ba da ƙayyadaddun sakamako. Kurakuran, gama-gari ga aikin NMT na farko, shine sikelin. Bisa ga ma'auni na yau, ƙirar ƙanana ne. Amfani da raka'a ƙananan kalmomi (Byte Pair Encoding) an ambaci su ta hanyar ambaton (Sennrich et al., 2015), amma babu zurfin bincike akan muhimmiyar rawar da yake takawa wajen sarrafa siffar Larabci a nan. Aikin daga baya, kamar na ƙungiyar Transformer ta Google (Vaswani et al., 2017), zai nuna cewa sikelin da ƙira (kulawar kai) suna haɓaka waɗannan fa'idodin na farko sosai.

Fahimta Mai Aiki: Ga masu aiki, wannan takarda tana nuna alama mai kyau. 1) Ba da fifiko ga NMT don Larabci: Ko da ƙirar asali sun dace da SMT kuma sun yi fice a cikin ƙarfin jurewa. 2) Kada a jefar da ilimin sarrafa kafin aiwatarwa: Fahimtar da al'ummar SMT suka samu game da rarraba kalmomi na Larabci har yanzu tana da mahimmanci. 3> Yi amfani da gama-gari: Sakamakon da ba na yanki ba shine ma'aunin mahimmanci don yiwuwar duniyar zahiri. Zuba jari na gaba ya kamata ya mai da hankali kan haɓaka wannan ta hanyoyin fasaha kamar fassarar baya (Edunov et al., 2018) da babban horon kafin aiwatarwa na harsuna da yawa (misali, mBART, M2M-100). Hanyar gaba a bayyane take: yi amfani da ikon gama-gari na ƙirar jijiyoyi, ciyar da ita tare da sarrafa kafin aiwatarwa mai ilimin harshe da babban bayanai, kuma a wuce daga kawai dacewa da SMT zuwa fiye da ita a cikin duk yanayi.

6. Tsarin Bincike & Nazarin Lamari

Tsarin Kimanta NMT don Harsuna Masu Ƙarancin Albarkatu/Masu Wadataccen Siffa:

1. Kafa Tushe: Kwatanta da ƙaƙƙarfan tushen SMT mai tushen jumla da aka daidaita (ba kawai tsarin da ba a ciki ba).
2. Rage Sarrafa Kafin Aiwatarwa Na Harshe: Gwada tasirin kowane mataki na sarrafa kafin aiwatarwa (daidaitawa, rarraba kalmomi, rarraba siffa) a keɓance kuma a haɗe.
3. Gwajin Damuwa Na Gama-gari: Kimantawa akan gwaje-gwajen da ba na yanki ba da yawa (labarai, kafofin sada zumunta, takaddun fasaha) don auna ƙarfin jurewa.
4. Binciken Kurakurai: Wuce BLEU. Rarraba kurakurai (siffa, tsarin kalmomi, zaɓin ƙamus) don fahimtar raunin ƙira na musamman ga harshen.

Nazarin Lamari: Amfani da Tsarin
Ka yi tunanin kimanta sabon ƙirar NMT don Swahili. Biye da wannan tsarin: 1) Gina tsarin Moses SMT a matsayin tushe. 2) Gwada matakan bincike daban-daban na siffa don sunaye da fi'ili na Swahili. 3) Gwada ƙirar akan rubutun labarai (cikin yanki), bayanan Twitter, da rubutun addini (waje da yanki). 4) Bincika idan mafi yawan kurakurai suna cikin haɗakar fi'ili (siffa) ko fassarar karin magana (kari). Wannan tsari mai tsari, wanda aka yi wahayi daga hanyar binciken wannan takarda, yana samar da fahimta mai aiki fiye da maki BLEU guda ɗaya.

7. Aikace-aikace Na Gaba & Hanyoyi

Binciken wannan aikin majagaba ya buɗe hanyoyi da yawa na gaba:

• Ci Gaban Ƙira: Amfani da ƙirar tushen Transformer (Vaswani et al., 2017) akan Larabci, waɗanda tun daga lokacin suka zama na zamani, mai yuwuwar samun mafi girman riba a daidaito da ƙarfin jurewa.
• Fassarar Harsuna Da Yawa & Sifili: Yin amfani da NMT na harsuna da yawa don inganta fassarar Larabci ta hanyar raba sigogi tare da harsunan da ke da alaƙa (misali, sauran harsunan Semitic) ko ta hanyar manyan ƙira kamar M2M-100 (Fan et al., 2020).
• Haɗawa da Ƙirar Harshe An Horar Da Su A Baya: Daidaita manyan ƙirar harshe guda ɗaya na Larabci (misali, AraBERT) ko na harsuna da yawa (misali, mT5) da aka horar da su a baya don ayyukan fassara, tsarin da ya kawo juyin juya hali ga aiki.
• Fassarar Larabci Na Karin Harshe: Miƙa NMT don ɗaukar ɗimbin bambancin yarukan Larabci, babban kalubale saboda rashin daidaitaccen rubutu da ƙarancin bayanai masu kama da juna.
• Aiwatarwa A Duniyar Zahiri: Ƙarfin jurewa da aka lura ya sa NMT ta zama manufa mai kyau don aikace-aikace na zahiri a cikin yanayi mai motsi kamar fassarar kafofin sada zumunta, chatbots na tallafin abokin ciniki, da fassarar labarai na ainihin lokaci.

8. Nassoshi

1. Bahdanau, D., Cho, K., & Bengio, Y. (2015). Fassarar injin neural ta hanyar koyon jeri da fassara tare. ICLR.
2. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Koyon wakilcin jumla ta amfani da mai rufewa-mai buɗewa na RNN don fassarar injin ƙididdiga. EMNLP.
3. Edunov, S., Ott, M., Auli, M., & Grangier, D. (2018). Fahimtar fassarar baya a sikelin. EMNLP.
4. Fan, A., Bhosale, S., Schwenk, H., Ma, Z., El-Kishky, A., Goyal, S., ... & Joulin, A. (2020). Bayan fassarar injin harsuna da yawa mai da Turanci a tsakiya. arXiv preprint arXiv:2010.11125.
5. Habash, N., & Sadat, F. (2006). Tsare-tsaren sarrafa kafin aiwatarwa na Larabci don fassarar injin ƙididdiga. NAACL.
6. Koehn, P., et al. (2003). Fassarar tushen jumla ta ƙididdiga. NAACL.
7. Sennrich, R., Haddow, B., & Birch, A. (2015). Fassarar injin neural na kalmomi da ba a saba gani ba tare da raka'a ƙananan kalmomi. ACL.
8. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Kulawa shine duk abin da kuke buƙata. NeurIPS.
9. Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., & Makhoul, J. (2014> Ƙirar haɗin gwiwar cibiyar sadarwar neural mai sauri da ƙarfi don fassarar injin ƙididdiga. ACL.