ICEfaces
  1. ICEfaces
  2. ICE-10447

mobi:camera - add HTML5 getUserMedia Support

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: EE-4.0.0.GA
    • Component/s: MOBI-Components
    • Labels:
      None
    • Environment:
      ICEmobile EE 1.3.1
    • Assignee Priority:
      P2
    • Affects:
      Compatibility/Configuration

      Description

      Enhancement of the ICEmobile Camera component with HTML5 getUserMedia support

          The Camera component will dynamically render JavaScript code for an HTML5 fallback mode when BridgeIt Native utility app is not available or not present
          The HTML5 fallback mode will do client-side detection of the getUserMedia API
              If the API is not present, the component will render an input file element as before
              If the API is present, the component will render a button to trigger the getUserMedia API and a simulated camera capture popup
          Simulated Camera Capture popup
              Will show the user the video capture stream from within the web application
              Will provide a capture button to take a photo
              Will provide a thumbnail of the captured image using an HTML5 canvas element
              Will allow the user to select the current image, or take another photo
              If possible, the image data may be stored in local storage until the form is posted
              The selected image will be serialized into the form so that it can be uploaded on the next form post
          The server-side mobi:camera component API will remain unchanged, if possible.

        Activity

        Hide
        Ken Fyten added a comment - - edited

        From Philip's commit log:

        Change file and canvas uploads to use FormData and submit to Auxillary upload post url.

        Show
        Ken Fyten added a comment - - edited From Philip's commit log: Change file and canvas uploads to use FormData and submit to Auxillary upload post url.
        Hide
        Ken Fyten added a comment -

        Testing using Jenkins ICEfaces 4 continuous build (http://dev.icesoft.com/jenkins/job/ICEfaces%204%20Trunk%20(Continuous)/1075/ / svn rvn# 44100)

        1. On desktop browsers, any file larger than 1MB fails immediately upon upload with a Network Connection Interrupted error. Smaller files work correctly.
          • The following JS errors are seen in the console:
            14:27:09.117 "[window] [Mon, 23 Feb 2015 21:27:09 GMT] the response does not contain XML data" bridge-support.js.jsf:1817
            14:27:09.118 "[window] [Mon, 23 Feb 2015 21:27:09 GMT] HTTP error [code: 0]: The Http Transport returned a 0 status code.  This is usually the result of mixing ajax and full requests.  This is usually undesired, for both performance and data integrity reasons." bridge-support.js.jsf:1817 
        2. On iphone 5 / iOS8.1.3, taking a picture works (via BridgeIt), the thumbnail shows, the upload works, but then the thumbnail is immediately cleared. If I recall correctly, I think it was needing to submit the thumbnail to the server so it could be rendered back out to the client after the submit. This is an app. developer choice. I think it is just as valid to clear the thumbnail after a successful submit, so we can leave it like this. As you mentioned in your commit log, it results in a much smaller sized submit to upload the photo without the thumbnail data.
        Show
        Ken Fyten added a comment - Testing using Jenkins ICEfaces 4 continuous build ( http://dev.icesoft.com/jenkins/job/ICEfaces%204%20Trunk%20(Continuous)/1075/ / svn rvn# 44100) On desktop browsers, any file larger than 1MB fails immediately upon upload with a Network Connection Interrupted error. Smaller files work correctly. The following JS errors are seen in the console: 14:27:09.117 "[window] [Mon, 23 Feb 2015 21:27:09 GMT] the response does not contain XML data" bridge-support.js.jsf:1817 14:27:09.118 "[window] [Mon, 23 Feb 2015 21:27:09 GMT] HTTP error [code: 0]: The Http Transport returned a 0 status code. This is usually the result of mixing ajax and full requests. This is usually undesired, for both performance and data integrity reasons." bridge-support.js.jsf:1817 On iphone 5 / iOS8.1.3, taking a picture works (via BridgeIt), the thumbnail shows, the upload works, but then the thumbnail is immediately cleared. If I recall correctly, I think it was needing to submit the thumbnail to the server so it could be rendered back out to the client after the submit. This is an app. developer choice. I think it is just as valid to clear the thumbnail after a successful submit, so we can leave it like this. As you mentioned in your commit log, it results in a much smaller sized submit to upload the photo without the thumbnail data.
        Hide
        Philip Breau added a comment -

        Ken,
        I can't reproduce the issue with the file upload. I've tested jenkins build #182 with Tomcat 7.0.42 (with post size limit increased) and Win 7 (Chrome 40, FF 35, IE 11), and Mac (Chrome 40 and Safari 8.0.3) and all them succeed processing and posting a 3.2mb jpg file. Have you tried different image files?

        I did notice that the post is including a large thumbnail data url. I'll fix that which should reduce the the post size by about half.

        Show
        Philip Breau added a comment - Ken, I can't reproduce the issue with the file upload. I've tested jenkins build #182 with Tomcat 7.0.42 (with post size limit increased) and Win 7 (Chrome 40, FF 35, IE 11), and Mac (Chrome 40 and Safari 8.0.3) and all them succeed processing and posting a 3.2mb jpg file. Have you tried different image files? I did notice that the post is including a large thumbnail data url. I'll fix that which should reduce the the post size by about half.
        Hide
        Ken Fyten added a comment -

        Sorry Philip, testing on svn rvn# 44097 I'm seeing the following failures:

        1. On desktop browsers, any file larger than 1MB fails immediately upon upload with a Network Connection Interrupted error. Smaller files work correctly.
        2. On iphone 5 / iOS8.1.3, taking a picture works (via BridgeIt), the thumbnail shows up, but uploading fails with "The uploaded image file could not be correctly processed."
        Show
        Ken Fyten added a comment - Sorry Philip, testing on svn rvn# 44097 I'm seeing the following failures: On desktop browsers, any file larger than 1MB fails immediately upon upload with a Network Connection Interrupted error. Smaller files work correctly. On iphone 5 / iOS8.1.3, taking a picture works (via BridgeIt), the thumbnail shows up, but uploading fails with "The uploaded image file could not be correctly processed."
        Hide
        Philip Breau added a comment -

        I removed the client side JPEG encoder and retested on all browsers. The image processing is now faster. On Chrome I recorded the following times for a jpg image of ~3.2mb:

        • load image file 28ms
        • create Image object 1000ms
        • create data URLs for thumbnail and image upload with canvs 8500ms

        This is noticably faster than using the jpeg encoder.

        But this creates a worse problem. The Canvas.toDataURL() implementations on all browsers default to creating an png file, and ignore the directive to encode to jpg, which would a reductiion in resolution for the thumbnail and a much smaller image file overall. The resulting png data url is approximately 10X the size of a jpg file. So a 3mb original jpg file will result in a 30mb png file. And uploading the 30mb png then locks up almost all browsers.

        So I don't think we have a choice but to scale larger image files. I've checked in a version that uses a scaling factor for images larger than 1mb and this seems to work well with reasonable performance on all desktop browsers. Scaling is done with Canvas and little loss of effective screen resolution is seen.

        Show
        Philip Breau added a comment - I removed the client side JPEG encoder and retested on all browsers. The image processing is now faster. On Chrome I recorded the following times for a jpg image of ~3.2mb: load image file 28ms create Image object 1000ms create data URLs for thumbnail and image upload with canvs 8500ms This is noticably faster than using the jpeg encoder. But this creates a worse problem. The Canvas.toDataURL() implementations on all browsers default to creating an png file, and ignore the directive to encode to jpg, which would a reductiion in resolution for the thumbnail and a much smaller image file overall. The resulting png data url is approximately 10X the size of a jpg file. So a 3mb original jpg file will result in a 30mb png file. And uploading the 30mb png then locks up almost all browsers. So I don't think we have a choice but to scale larger image files. I've checked in a version that uses a scaling factor for images larger than 1mb and this seems to work well with reasonable performance on all desktop browsers. Scaling is done with Canvas and little loss of effective screen resolution is seen.

          People

          • Assignee:
            Philip Breau
            Reporter:
            Philip Breau
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: