[Freeswitch-users] need help using http_cache + s3

Fri Jun 17 05:29:15 MSD 2016

OK, I got it working - tweaked a bunch of things so not sure what was the original problem, but for posterity here is the working config:

——http_cache.conf.xml—

    <profiles>
    <!-- amazon s3 security credentials -->
    <profile name="s3">
      <aws-s3>
        <!-- 20 character key identifier -->
        <access-key-id><![CDATA[my-access-key-id]]></access-key-id>
        <!-- 40 character secret -->
        <secret-access-key><![my-secret-access-key]]></secret-access-key>
      </aws-s3>

      <!-- optional list of domains that this profile will automatically be applied to -->
      <!-- if you wish to apply the s3 credentials to a domain not listed here, then use
           {profile=s3}http://foo.s3... -->
      <domains>
        <domain name=“my-bucket.s3.amazonaws.com"/>
      </domains>
—————————

———dialplan———

   <extension name="announcement">
      <condition field="${sip_req_user}" expression="^annc$"/>
      <condition field="${sip_req_params}" expression="^play=(.*)$">
        <action application="answer"/>
        <action application="sleep" data="1000"/>
        <action application="playback" data="http_cache://http://my-bucket.amazonaws.com/$1"/>
        <action application="hangup"/>
      </condition>
   </extension>

------------------------
On Jun 16, 2016, at 3:42 PM, Christopher Rienzo <cmrienzo at gmail.com> wrote:

Base-domain stuff is only if you host your own S3-compatible service.  It will default to amazonaws.com <http://amazonaws.com/> if not set.  If setting profile=s3 doesn't work, then something is really broken in the config.

On Thu, Jun 16, 2016 at 3:32 PM, Dave Horton <daveh at beachdognet.com <mailto:daveh at beachdognet.com>> wrote:
Well, from looking at the code, part of the problem is that there is an additional — undocumented — configuration parameter called ‘base-domain’ which is required for the AWS stuff to kick in….still trying to figure out how this works, by reading the code……anyone who has knowledge please chime in

On Jun 16, 2016, at 2:42 PM, Christopher Rienzo <cmrienzo at gmail.com <mailto:cmrienzo at gmail.com>> wrote:

Given the example configuration, you can remove {profile=s3}http_cache:// from your playback URL.  Also, you can change the config / reload the module / and test using fs_cli http_get for rapid verification of your setup.

On Thu, Jun 16, 2016 at 2:13 PM, Dave Horton <daveh at beachdognet.com <mailto:daveh at beachdognet.com>> wrote:
I’m having trouble using http_cache to download and play a wave file from my amazon S3 bucket.
I’m following the basic config at https://freeswitch.org/confluence/display/FREESWITCH/mod_http_cache <https://freeswitch.org/confluence/display/FREESWITCH/mod_http_cache> but I must be missing something.

My http_cache.conf.xml looks like this:

———————————
<configuration name="http_cache.conf" description="HTTP GET cache">
  <settings>
    <!-- set to true if you want to enable http:// and https:// formats.  Do not use if mod_httapi is also loaded -->
    <param name="enable-file-formats" value="true"/>
    <param name="max-urls" value="10000"/>
    <param name="location" value="$${cache_dir}"/>
    <param name="default-max-age" value="86400"/>
    <param name="prefetch-thread-count" value="8"/>
    <param name="prefetch-queue-size" value="100"/>
    <!-- absolute path to CA bundle file -->
    <param name="ssl-cacert" value="$${certs_dir}/cacert.pem"/>
    <!-- verify certificates -->
    <param name="ssl-verifypeer" value="true"/>
    <!-- verify host name matches certificate -->
    <param name="ssl-verifyhost" value="true"/>
    <!-- default is 300 seconds, override here -->
    <!--param name="connect-timeout" value="300"/-->
    <!-- default is 300 seconds, override here -->
    <!--param name="download-timeout" value="300"/-->
  </settings>

    <profiles>
    <!-- amazon s3 security credentials -->
    <profile name="s3">
      <aws-s3>
        <!-- 20 character key identifier -->
        <access-key-id><![CDATA[my-access-key-here]]></access-key-id>
        <!-- 40 character secret -->
        <secret-access-key><![CDATA[my-secret-access-key-here]]></secret-access-key>
      </aws-s3>
      <!-- optional list of domains that this profile will automatically be applied to -->
      <!-- if you wish to apply the s3 credentials to a domain not listed here, then use
           {profile=s3}http://foo.s3. <http://foo.s3./>.. -->
      <domains>
        <domain name=“my-bucket.s3.amazonaws.com <http://my-bucket.s3.amazonaws.com/>"/>
      </domains>
    </profile>
   </profiles>
</configuration>
—————————————

My dialplan looks like this:

——————————————
   <extension name="announcement">
      <condition field="${sip_req_user}" expression="^annc$"/>
      <condition field="${sip_req_params}" expression="^play=(.*)$">
        <action application="log" data=“file to play: $1"/>
        <action application="answer"/>
        <action application="sleep" data="1000"/>
        <action application="playback" data="{profile=s3}http_cache://https://my-bucket.s3.amazonaws.com:/$1 <https://my-bucket.s3.amazonaws.com/$1>"/>
        <action application="hangup"/>
      </condition>
   </extension>
_______________________

The log file says this:

   EXECUTE sofia/drachtio_mrf/5083084809 at 68.64.80.16 <mailto:5083084809 at 68.64.80.16> playback({profile=s3}http_cache://http://wcs-icr.s3.amazonaws.com:/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav <http://wcs-icr.s3.amazonaws.com/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav>)
   2016-06-16 14:02:59.173718 [DEBUG] mod_http_cache.c:569 Locked cache
   2016-06-16 14:02:59.173718 [INFO] mod_http_cache.c:661 Cache MISS: size = 0 (0 MB), hit ratio = 0/1
   2016-06-16 14:02:59.173718 [DEBUG] mod_http_cache.c:726 Adding http://wcs-icr.s3.amazonaws.com:/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav(/usr/local/freeswitch/cache/b6/456219-e1eb-43c2-bbd9-449cfb3af6ac.wav) <http://wcs-icr.s3.amazonaws.com/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav(/usr/local/freeswitch/cache/b6/456219-e1eb-43c2-bbd9-449cfb3af6ac.wav)> to cache index 0
   2016-06-16 14:02:59.173718 [DEBUG] mod_http_cache.c:580 Unlocked cache
   2016-06-16 14:02:59.173718 [DEBUG] mod_http_cache.c:1077 opening /usr/local/freeswitch/cache/b6/456219-e1eb-43c2-bbd9-449cfb3af6ac.wav for URL cache
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 HTTP/1.1 403 Forbidden
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 x-amz-request-id: F27F3491740AC96E
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 x-amz-id-2: 1LII609UCqdosrj+3tha2MrWfkoDjIgJ/wK2V47LDkgpn6/i8V4GFRxVzzoXRj9Rzbq6BMP15Po=
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 Content-Type: application/xml
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 Transfer-Encoding: chunked
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 Date: Thu, 16 Jun 2016 18:02:58 GMT
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548 Server: AmazonS3
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:548
   2016-06-16 14:02:59.233717 [ERR] mod_http_cache.c:1131 Received HTTP error 403 trying to fetch http://wcs-icr.s3.amazonaws.com:/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav <http://wcs-icr.s3.amazonaws.com/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav>
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:569 Locked cache
   2016-06-16 14:02:59.233717 [INFO] mod_http_cache.c:683 Failed to download URL http://wcs-icr.s3.amazonaws.com:/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav <http://wcs-icr.s3.amazonaws.com/RBT-5762afa9991351886b295fbc/prompt/1466086249630-call-blocked.wav>
   2016-06-16 14:02:59.233717 [DEBUG] mod_http_cache.c:580 Unlocked cache
   EXECUTE sofia/drachtio_mrf/5083084809 at 68.64.80.16 <mailto:5083084809 at 68.64.80.16> hangup()

————————

I’ve fiddled with a bunch of different things, but no luck.  I took a tshark of the outgoing http get and I can see it is not putting in the headers to authenticate, so it seems it is not getting into
the S3 code in http_cache.  The docs do not include a complete S3 working example, unfortunately.

Any help or pointers from those who have gotten this working would be appreciated…

Dave
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org <mailto:consulting at freeswitch.org>
http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/>

Official FreeSWITCH Sites
http://www.freeswitch.org <http://www.freeswitch.org/>
http://confluence.freeswitch.org <http://confluence.freeswitch.org/>
http://www.cluecon.com <http://www.cluecon.com/>

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users>
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users>
http://www.freeswitch.org <http://www.freeswitch.org/>
_________________________________________________________________________
Professional FreeSWITCH Consulting Services: 
consulting at freeswitch.org <mailto:consulting at freeswitch.org>
http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/>

Official FreeSWITCH Sites
http://www.freeswitch.org <http://www.freeswitch.org/>
http://confluence.freeswitch.org <http://confluence.freeswitch.org/>
http://www.cluecon.com <http://www.cluecon.com/>

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users>
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users>
http://www.freeswitch.org <http://www.freeswitch.org/>

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org <mailto:consulting at freeswitch.org>
http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/>

Official FreeSWITCH Sites
http://www.freeswitch.org <http://www.freeswitch.org/>
http://confluence.freeswitch.org <http://confluence.freeswitch.org/>
http://www.cluecon.com <http://www.cluecon.com/>

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users>
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users>
http://www.freeswitch.org <http://www.freeswitch.org/>

_________________________________________________________________________
Professional FreeSWITCH Consulting Services: 
consulting at freeswitch.org
http://www.freeswitchsolutions.com

Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20160616/56de4ca2/attachment-0001.html 