I looked at my Deutsch-Lateinische Phraseology, one from the 18th century, the other from 2015 (here’s a translation), and I can’t find an entry for “to be cold”. There’s “freezing” as in "The river froze, so that one could drive carts over it). There’s also algeo “I suffer from the cold”.
Now, meus non fui frigus is nonsense because i) meus is the possessive pronoun 1st person sg. nominative masculine. ii) frigus, -oris is neuter. iii) fui is 1st person sg. If you want to force a meaning, it’s gonna be “in respect to the cold, I was not my own person,” but the accusative of respect is very uncommon and an imitation of a greek idiom (where it’s very common).
In general, Latin expresses states of body and mind as verbs (" esurio I hunger, sitio I thirst") or with the genitive or ablative of an abstractum (bona valetudine esse “to be of good health”, morbo mori “to die of sickness (morbus)”
To express a possession or affliction, the usual way is to use the dative: mihi est hortus “I have a garden (there a garden for me)”. Isti viro pauci sunt capilli “That man here’s got few hairs (there are few hairs for that man)”
I would recommend to you, that now that you have a decent vocabulary, you try to memorise basic declension (o-stems, a-stems, and three or four common nouns of the 3rd declension, and most of all, verbs. Don’t bother with the details of the Perfect, stick with the Present active, for now. sum, es, est, sumus, estis, sunt you should also know by heart. Then, there’s the regular paradigms of laborare, laboro, delere, deleo, and the slightly irregular so-called consonant stems of capere, capio and petere, peto and the like.