Ai Face-Markerless Facial Mocap For Blender

by Legacy developers in Addons


Item Rating

This item has an average rating of 3 from 5 ratings by the community.

0%
40%
20%
0%
40%
  • hari
    over 2 years ago

    4 stars for This product. Its good low budget very effective.. 5 stars for the developer  he is very responsible he will tell whatever the doubts you have..I asked him out of the box also  he helped me in edit mode also... thankyou so much sir.

  • Benjamin Andermatt
    almost 3 years ago

    I think, overall this is a really impressive software.
    I've played around with it for a day now, and although, I don't think I can really benefit from it in the current stage, I see the basis for something great here.
    The tool DOES what is shown in the video (contrary to what others have written). I did experience quite a few crashes, but I did love the simplicity and user-friendliness of the interface.
    The main problem is, that at this time the result just isn't precise enough. It can detect the broader movements of the mouth and eyebrows, but no details. This means it can't be used for speech, or any emotions more nuanced than a big grin. Since there's also no head tracking, this sadly means the tool doesn't really take away any work for the animator.
    I'm not exactly sure at what point the loss in quality happens though. The tracking points on the face look very precise, but they don't translate too well to the blend-shapes. Maybe there just need to be more of them? For example for mouth shapes common in speaking, like F and S sounds.
    I'd also love to see support for higher frame-rates, as facial movements can be very quick and subtle.
    Lastly it wasn't very clear to me how the training expressions would be used - Do I smile with just my mouth or with my whole face? Does it matter? Can anyone really lift only the outer part of their eyebrows except Jack Nicholson?
    But after all, this tool is still pretty new, and I'm excited to see where it goes from here.

    • Onkar Gaikar

      almost 3 years ago

      Thank you Benjamin for your detailed review. Can you please tell me more about how exactly the program crashes? Also, during training smile just checks the distance between corners of lips. And outer eyebrow means you can lift your whole eyebrow. if you got some more questions, please feel free to contact me using the "Ask a Question" button. Also, I am very glad to tell you that the next update of the AI face will also calculate head rotations and it will be in real-time! If you want some more features, please contact me anytime.

  • Nate
    about 3 years ago

    Alright, an actual review with suggestions, unlike these other jokers expecting the animation to automatically work. I don’t, however, think it’s a 25 NZD addon just yet. Maybe more like 10 or 15. It’s a great animation, and it has massive potential (and honestly, the AI is incredible), but in this early state it needs a bit more work. If improved on I can honestly see this becoming a easy 50 NZD product.

    First thing: DON'T BUY THIS EXPECT IT TO AUTOMATICALLY WORK, as if you don’t do the proper set up for the face (adding vertex groups and such) the animations will simply will not be applied. The application does work, it’s just the idiots who have 1 star voted this into the ground haven’t properly added the vertex groups.

    This also leads into my first suggestion for the application: add a feature which allows for mesh of faces to have the Vertex groups automatically applied. Something similar to Autorig pro perhaps, I’m unsure. It’ll sure save this addon from the dummies who think it’ll work out the door. Hell, straight up integration with Autorig pro and rigify could be good, since I know a lot of people who use those addons.

    Second thing: In my testing it seemed the addon worked way better with low res videos. I assume this is because it was built using a low end Camera, as you see in the trailer and tutorial video on the Developer’s YouTube channel. Manually shrinking the res of your captured video can work wonders for the quality of the output, just remember to train the AI to your face at first.

    Second suggestion: Something that automatically does this, again if the user wants.

    Third thing: This addon is honestly surprisingly fast to use. It takes about 10-15 minutes on my Ryzen 5 1600 to do just under 2000 frames. It doesn’t even use all the CPU so you can do stuff in the background as well. I kind of wonder how fast it could go using a GPU, but that’ll require some work. Importing the animation file into blender is quick and easy as well, although 3 times now I’ve tried clicking directly on the csv. file. To be honest, I really don’t have much of a problem with the software at all, or the addon. It’s just a few nicks and knacks I don’t really like or could be improved on.

    Alright, some spare suggestions:

    - Can you capture eye movement? I’d assume vertex groups for it would be easy, have the application able to record the eye’s x and y relatively to the center of the eye-lid markers, and set looking straight forwards in Blender on “Eye x axis” and “Eye y axis” to 0.5 each, more towards 0 makes the eyes look up or to the left, more towards 1 makes the eyes look down or to the right. Bit unconventional but it could work.

    - The documentation needs a bit of work. In the section “Importing Data Files” you should specifically mention the fact you need to manually add Vertex groups onto your own custom character. You should also mention this on the main page for BlenderMarket.

    - I’m unsure of how the ai works, I’m not much for code myself, but perhaps if the AI could randomly generate faces for the actor to pull it could train the AI more accurately, maybe if someone wants a more accurate result. As it stands now, the AI is pretty amazing already.

    - In the promotional material, don’t use comic sans text font. This is an amazing application, comic sans isn’t doing it any favours.

    My testing of downscaling the sample video to see how different results looked: https://youtu.be/GfRF1K5yDg8

    • Onkar Gaikar

      about 3 years ago

      Thank you Nate for your detailed review. I will consider your all suggestions in the next update. In the next few updates, this tool will definitely improve a lot. Also, you can get better results with good lighting.
      If you want to share some more suggestions with me feel free to contact me using the "Ask a Question" button.

  • lucas
    about 3 years ago

    does not deliver what was promised in the video.

    • Onkar Gaikar

      about 3 years ago

      Please contact me if you are facing any difficulties using the ask a question button. I will clear all of your doubts. And about the things shown in the video, let me tell you my deep learning model was not trained on my own face. But still,, it was managed to predict all shape keys. Let me assure you if ai is detecting your face accurately then you don't have to worry. Please send me your problem in detail (with screenshots or your screen recording so I could see what you are doing wrong) I won't take too much time to respond and also if you managed to use it in the right way on your own don't forget to update your rating!

  • drfugazi
    about 3 years ago

    Not working for me. Video caption records a video but when trying to use the Addon it just happens nothing.

    • Onkar Gaikar

      about 3 years ago

      Please provide details of your problem in detail. Use "Ask a Question" button to contact me. And also if your system is recording video properly and the system is detecting your face accurately then there is no problem. And also read FAQs, it will clear most of your doubts. Please contact me anytime.

  • Displaying 5 ratings

Purchase this item to leave a review!

Choose a product version:

Sales 200+
Customer Ratings 5
Average Rating
Dev Fund Contributor
Published about 3 years ago
Blender Version 4.0, 3.6, 3.5, 3.4, 3.3, 3.2, 3.1, 3.0, 2.93, 2.92, 2.91, 2.9, 2.83, 2.82, 2.81, 2.8
License GPL
Have questions before purchasing?

Contact the Creator with your questions right now.

Login to Message