arvind

@arvind

active 3 weeks, 2 days ago
  • arvind posted an update in the group Group logo of Deep learningDeep learning 5 months, 4 weeks ago

    Notes from NLP meetup

    BERT
    GPT2
    XLNET
    CTRL
    TransformerXL
    ERNIE
    Roberta
    Albert
    Distillbert

    HuggingFace

    Glue Benchmark
    IMDB dataset sentiment classification
    SQUAD
    SuperGlue

    Google search uses BERT – going from keyword to NLU
    Semantic role labelling

    From F to A on NY Regents Science Exams : An overview of Aristo Project BERT and Roberta on class 8 science exams

    How is BERT trained

    MLM – masked language model
    NSP – next sentence prediction


    Word2vec visualisation
    —-
    DistillBERT for sentiment analysis

    Model | Tokenizer | Pretrained-weights-shortcut

    GPT is more suited for NLG than BERT

    Dummy classifier

    ULMFit 60K dictionary size

Skip to toolbar