EU Citizens and Non-EU Spouses and Relatives 

My husband belongs to an EU country, but I do not. I know my spouse is entitled to free entry and freedom of movement. I suppose being married to him will enable me to have the same rights. Is that correct?

Yes, that is correct! You have the same rights as your husband. Your status is generally dependent upon marriage. Especially, in your case it would be wise to register at the foreigners office in order to get a "residence card (Aufenthaltskarte)". This stamp will make sure that no (police) officer comes up with the idea you are living here illegally. N.B. This is not a permission granted (Aufenthaltsgenehmigung) but just a formal acknowledgment of your status.
§

Do my relatives from non-European countries have the right to follow me to Germany?

Supposing you, as a European, are gainfully employed (self-employed as well as employees) then you are entitled to have your closest family members follow you to Germany. The same goes for your spouse even if he or she is from a third-land country. These persons are:

  • spouse,
  • (grand)children younger than 21, (grand)parents.
    §
Do any formalities have to be met or can I just move in?

Oh, yes indeed. You have to show that you have enough funds to sustain your life and sufficient health insurance.
§

Do I have to apply for a visa when entering Germany from outside Schengen Area?

Yes, unfortunately you will have to apply for a visa. However, if you have clean records, it should be no problem.
§

What does it mean to have one’s livelihood secured? Don’t tell me I have to have as much money as Rockefeller.

Your livelihood is secured if you can yourself meet the cost of living, including sufficient health insurance coverage, without having to make a claim on public funds. In numbers this means: € 345 in Germany’s older states (including all Berlin) and € 331 in the Germany’s newer states (former GDR) and on top of this go the rental costs. In other words, an apartment for € 400 will mean in Hamburg € 745 and in Dresden € 731. N.B. These values are the absolute minimum!
§

How do I prove that my means of livelihood are sufficient and secured?

This can be bank statements, credit cards, tax auditor’s statement of interest, of dividends, etc.