Historically, there was a lot of resistance to the Agama-s from the vedAntins. There were even texts which said that the pAncarAtrins should not be invited to partake in SrAddha meals, people should not talk to them, etc., alongside the texts that supported the Agama adherents. There was also counter-attack from the pAncarAtra camp, including statements calling the veda-s as perverted texts incapable of fulfilling the human values (purushArthAproyojaka).
It is in this atmosphere that SrI yAmuna-muni undertook his valiant defense of the pAncarAtra through his work Agama-prAmANya, to bring about reconciliation between the two camps. He pointed out that the pAncarAtra and the veda-s both originated from SrIman nArAyaNa, and were complementary to each other. Inevitably it was a very sensitive undertaking, since it touched on the sensitivities of both the opposing camps. Then there came the time when the pAncarAtrins claimed superiority over the veda-s saying that the pAncarAtra was more ancient, and the veda-s came later. Perhaps in this atmosphere, SrI vedAnta-deSika endeavored to bring a balance between the veda-s and the Agama-s, quoting passages from the pAncarAtra text Lakshmi-tantra) that the wise man should never transgress even in his thoughts the conduct prescribed in the veda-s. It cannot but be noticed that our great pUrvAcArya-s have done exceptional service in bringing about the reconciliation between the vedanta adherents and the Agama group, trying to make sure that people don't succumb to self-destruction through disunity.
Even among the pAnca-rAtrins, it appears that there was not clear homogeneity. Four main divisions are recognized in many samhitA-s. Even though all agree that vAsudeva is the Supreme Spirit and that salvation is to be got by worshipping him, they differ in the methods of worship, the benefits that accrue from the worship, the methods of rituals etc. Mix-up of movement from one group to another was frowned upon (pAdma samhitA).
These four divisions are called siddhAnta-s. As mentioned in the pAdma samhitA, these are 1) mantra siddhAnta, Agama siddhAnta, tantra siddhAnta, and tantrantara siddhAnta. SrI vedAnda-deSika in his pAncarAtra rakshA quotes from the hayagrIva samhitA that Agama siddhAnta is meant to accomplish salvation alone, tantra siddhAnta is for accomplishing all four values of life (wealth, virtue, pleasures, and salvation), and tantrAntara siddhAnta will fulfil all that one desires.
The four siddhAnta-s differ in the number of forms they worship, the way they consecrate the iconic forms in the temples, the number of forms of deities that are worshipped (e.g., one form - the deity is not specified; four forms - vAsudeva, sankarshNa, pradyumna, and aniruddha; nine forms - in addition to the above four, nArAyaNa, hayagrIva, vishNu, nr*simha, and varAha) etc. The differences are described in great detail in the referenced texts.
Other Opposition to pA~NcarAtra:
Other main points raised by the vedAntin-s were:
1. The pA~ncarAtra prescribed, or required, a special initiation called dIkshA for anyone to be qualified for performing temple worship (the equivalent of priests in the temple). dIkshA was not explicitly mentioned in the veda-s, and so the argument was that pA~ncarAtra was non-vedic.
2. Unlike the pA~ncarAtra system whose authorship was not ascribable to any human source, the vaikhAnasa school was initiated by a vedic sage by name vaikhAnasa, and so it was easier to accept the vaikhAnasa system's known vedic basis.
3. A very well known mImAmasaka by name kumArilabhaTTa (around 800 A.D.) published his work titled tantravArtikA, in which he included the pA~ncarAtra as a non-vedic Agama. Because of his popularity, the confidence of the people in the vedic origin of pA~ncarAtra was put under severe test.
4. The interpretation of four aphorisms of the brahma sUtra-s considered to deal with the pA~ncarAtra system by the well-known Adi Sankara led him to conclude that the pA~ncarAtra was non-vedic. (See /Views for the details of this).
5. In one of the pA~ncarAtra samhitA-s (parama samhitA), SANDilya is quoted as saying that he has studied all the four veda-s but has not been able to find beyond all doubt the road to bliss, and so he undertook the study of pA~ncarAtra. This has been quoted by the advaitin-s as proof that the pA~ncarAtra undermines the veda-s by saying that they are incapable of revealing the truth.
6. The term sAttvata refers to a class of low-born people called vrAtya-s according to manu-smr*ti. Thr vrAtya community is decidedly non-vedic. Since the pA~ncarAtrin-s are referred to as sAttvata-s, they must be the same class of low-borns that is referred to in the smr*ti, and so pA~ncarAtra is non-vedic, and the bhAgavata-s (sAttvata-s) are not even brAhmaNa-s.
7. Partaking the food offered in worship is forbidden in smr*ti, and since the people who do temple worship in the pA~ncarAtra tradition partake in the food offered to the Lord, they are in violation of smr*ti-s.
8. The people who do temple worship should not benefit by this activity, and some of the people who do pA~ncarAtra worship have their livelihood from this.
9. vAsudeva, the person from whom the pA~ncarAtra is said to have originated, is known to have deceived the asura-s in His mohini incarnation, deceived mahAbali in His trivikrama incarnation, etc., and so He is unreliable, and could have also deceived people by creating the pA~ncarAtra.
10. While the veda-s have got a particular sequence (krama) - I presume the reference here is to the intonations , there is no such krama for pA~ncarAtra texts, and so pA~ncarAtra is non-vedic in character.
11. For the same rite, there is a vedic and a tantric rite, and this again establishes that Agama-s are opposed to, and different from the veda-s.