bert-for-tf2 · PyPI
pypi.org › project › bert-for-tf2Jan 21, 2021 · Usage. BERT in bert-for-tf2 is implemented as a Keras layer. You could instantiate it like this: from bert import BertModelLayer l_bert = BertModelLayer (** BertModelLayer. Params (vocab_size = 16000, # embedding params use_token_type = True, use_position_embeddings = True, token_type_vocab_size = 2, num_layers = 12, # transformer encoder params hidden_size = 768, hidden_dropout = 0.1 ...