Re: As a matter of interest ....
The 64 core parallella chip seems to be about to start production
Actually, it's not. The original Kickstarter campaign included the 64-core boards as a stretch goal, which was not met. Those 64-core boards they've been testing are engineering prototypes, only going to backers who came in at a certain level.
The whole Parallella project has been something of a disappointment, IMO. I think they over-promised (at least the 16-core machine isn't really a "supercomputer for everyone") and struggled to deliver. At least we know they've been plugging away at trying to make it a success and I do have sympathy for them in terms of the unforeseen problems they ran into. They have delivered at least some 16-core boards and hopefully they'll get around to delivering the rest to all the Kickstarter and pre-order customers within the next month. I'm one of the pre-order customers, so I'm hoping that they'll clear their commitments to everyone who ordered one within that time frame.
After that, and people have the boards in hand, hopefully people will still have enough interest in the platform for them to be able to make money by ramping up to full-scale production of the 16-core boards... I'm sure they're still doing work on the 64-core (and higher, up to 1024-core) and if they can get the funding for it, that's where they do want to go. I just don't expect it any time soon...
As for your idea of neural nets, I'm sure that it's pretty feasible to run them on the Epiphany cores. There's a pretty long thread about it on their forums somewhere. It's nowhere near the level of brain simulation, of course, but you can always cluster them and even single boards should be pretty efficient, given the right algorithms and such.