1. Did you defined flyGrid.columns.NestedColumns? Relating to Virtual Nested Grids sample, virtualNestedGridForm.FillNestedGridsGrid initilize 3 levels of columns by InitColumns method.
2. If you're using VirtualMode_InitNewNode event handler, you should return VirtualNestedGridNode inheritor - see the Virtual Nested Grids sample,
virtualNestedGridForm.nestedGrid_VirtualMode_InitNewNode method returns CheckedVirtualNestedGridNode that inherits from VirtualNestedGridNode.