It's neat to see the new primitives getting picked up, but at the same time the original problem doesn't seem to motivate it.

Instead of a single neural model per material, why not pass in a small vector summarizing the material properties besides the U,V,normal,etc properties for the pixel? A single model for all the materials would presumably be more compactly represented compared to one per material type: there is opportunity for compression in common properties of the materials.