Link

Automatic Prompt Generation in PET

Label to Word Mapping

\[\begin{gathered} \mathcal{V}^c=\underset{v\in\mathcal{V}}{\text{Top-}k}\bigg\{ \sum_{x_{in}\in\mathcal{D}_{train}^c}{ \log{P_\theta\Big( \text{[MASK]}=v|\mathcal{T}(x_{in}) \Big)} } \bigg\}, \\ \text{where }\mathcal{D}_{train}^c\subset\mathcal{D}_{train}\text{ is the subset of all examples of class }c. \end{gathered}\]

Prompt Generation

\[\begin{aligned} (x_1,y)&\Rightarrow\text{<}p_1\text{>}\mathcal{M}(y)\text{<}p_2\text{>}\text{<}x_1\text{>} \\ (x_1,y)&\Rightarrow\text{<}x_1\text{>}\text{<}p_1\text{>}\mathcal{M}(y)\text{<}p_2\text{>} \\ (x_1,x_2,y)&\Rightarrow\text{<}x_1\text{>}\text{<}p_1\text{>}\mathcal{M}(y)\text{<}p_2\text{>}\text{<}x_2\text{>} \end{aligned}\] \[\begin{gathered} (\text{"Best pizza I ever had."},positive)\Rightarrow\text{"Best pizza I ever had. It was }great\text{ !"} \end{gathered}\] \[\begin{gathered} \hat{\mathcal{T}}=\underset{\mathcal{T}\in\text{T}}{\text{argmax}}{ \sum_{x_{in},y\in\mathcal{D}_{train}}{ \log{ P_\theta\Big( \text{[MASK]}=\mathcal{M}(y)|\mathcal{T}(x_{in}) \Big) } } } \end{gathered}\]

Prompt Ensemble